[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786008495763054-20932-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 09:28:15 dist-test-slave-45dx krb5kdc[20947](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 09:28:15 dist-test-slave-45dx krb5kdc[20947](info): set up 2 sockets
Aug 06 09:28:15 dist-test-slave-45dx krb5kdc[20947](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260806 09:28:17.238025 20932 mini_kdc.cc:121] Time spent starting KDC: real 1.372s user 0.002s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786008495763054-20932-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786008495763054-20932-0/krb5kdc/test-user.keytab.
Aug 06 09:28:17 dist-test-slave-45dx krb5kdc[20947](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008497, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T09:28:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T09:28:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.113.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.113.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.113.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab.
Entry for principal kudu/127.20.113.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab.
I20260806 09:28:17.452845 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786008495763054-20932-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786008495763054-20932-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786008495763054-20932-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.113.62:41769
--webserver_interface=127.20.113.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab
--principal=kudu/127.20.113.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:40377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.113.62:41769
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:28:17.828990 20965 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:28:17.829296 20965 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:28:17.829483 20965 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:28:17.837291 20965 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 09:28:17.837409 20965 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:28:17.837507 20965 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:28:17.837605 20965 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 09:28:17.837690 20965 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 09:28:17.848011 20965 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:40377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786008495763054-20932-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786008495763054-20932-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.113.62:41769
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab
--principal=kudu/127.20.113.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.113.62:41769
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.113.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786008495763054-20932-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:28:17.849447 20965 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:28:17.850919 20965 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:28:17.864069 20972 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:28:17.863935 20980 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:28:17.863981 20974 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:28:17.864333 20965 server_base.cc:1034] running on GCE node
I20260806 09:28:17.865653 20965 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:28:17.867131 20965 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:28:17.868414 20965 hybrid_clock.cc:648] HybridClock initialized: now 1786008497868364 us; error 79 us; skew 500 ppm
I20260806 09:28:17.868799 20965 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:28:17 dist-test-slave-45dx krb5kdc[20947](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008497, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:28:17.872246 20965 init.cc:377] Logged in from keytab as kudu/127.20.113.62@KRBTEST.COM (short username kudu)
I20260806 09:28:17.873828 20965 webserver.cc:460] Webserver started at http://127.20.113.62:38227/ using document root <none> and password file <none>
I20260806 09:28:17.874539 20965 fs_manager.cc:359] Metadata directory not provided
I20260806 09:28:17.874643 20965 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:28:17.874929 20965 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:28:17.879175 20965 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786008495763054-20932-0/minicluster-data/master-0/data/instance:
uuid: "1f6faea2cc554242a8bb0af31de5aee1"
format_stamp: "Formatted at 2026-08-06 09:28:17 on dist-test-slave-45dx"
server_key: "9fae3f15f8c72e0703e6a321bba67f9c"
server_key_iv: "1abe2decd051891d221ab1c7907f5399"
server_key_version: "encryptionkey@0"
I20260806 09:28:17.879943 20965 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786008495763054-20932-0/minicluster-data/master-0/wal/instance:
uuid: "1f6faea2cc554242a8bb0af31de5aee1"
format_stamp: "Formatted at 2026-08-06 09:28:17 on dist-test-slave-45dx"
server_key: "9fae3f15f8c72e0703e6a321bba67f9c"
server_key_iv: "1abe2decd051891d221ab1c7907f5399"
server_key_version: "encryptionkey@0"
I20260806 09:28:17.885432 20965 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260806 09:28:17.888926 20989 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:28:17.890282 20965 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260806 09:28:17.890427 20965 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786008495763054-20932-0/minicluster-data/master-0/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786008495763054-20932-0/minicluster-data/master-0/wal
uuid: "1f6faea2cc554242a8bb0af31de5aee1"
format_stamp: "Formatted at 2026-08-06 09:28:17 on dist-test-slave-45dx"
server_key: "9fae3f15f8c72e0703e6a321bba67f9c"
server_key_iv: "1abe2decd051891d221ab1c7907f5399"
server_key_version: "encryptionkey@0"
I20260806 09:28:17.890584 20965 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786008495763054-20932-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786008495763054-20932-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786008495763054-20932-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:28:17.904960 20965 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:28:17.909283 20965 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:28:17.909515 20965 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:28:17.930084 20965 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.62:41769
I20260806 09:28:17.930109 21067 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.62:41769 every 8 connection(s)
I20260806 09:28:17.931598 20965 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
I20260806 09:28:17.932781 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 20965
I20260806 09:28:17.933180 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786008495763054-20932-0/minicluster-data/master-0/wal/instance
I20260806 09:28:17.935092 20932 external_mini_cluster.cc:1428] Setting key b584153fd2ed042d29cc890b918c55b6
I20260806 09:28:17.935807 21068 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:28:17.948002 21068 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f6faea2cc554242a8bb0af31de5aee1: Bootstrap starting.
I20260806 09:28:17.951229 21068 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1f6faea2cc554242a8bb0af31de5aee1: Neither blocks nor log segments found. Creating new log.
I20260806 09:28:17.952394 21068 log.cc:824] T 00000000000000000000000000000000 P 1f6faea2cc554242a8bb0af31de5aee1: Log is configured to *not* fsync() on all Append() calls
I20260806 09:28:17.955225 21068 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f6faea2cc554242a8bb0af31de5aee1: No bootstrap required, opened a new log
I20260806 09:28:17.962826 21068 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1f6faea2cc554242a8bb0af31de5aee1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f6faea2cc554242a8bb0af31de5aee1" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 41769 } }
I20260806 09:28:17.963267 21068 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1f6faea2cc554242a8bb0af31de5aee1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:28:17.963389 21068 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1f6faea2cc554242a8bb0af31de5aee1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f6faea2cc554242a8bb0af31de5aee1, State: Initialized, Role: FOLLOWER
I20260806 09:28:17.964062 21068 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1f6faea2cc554242a8bb0af31de5aee1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f6faea2cc554242a8bb0af31de5aee1" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 41769 } }
I20260806 09:28:17.964401 21068 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1f6faea2cc554242a8bb0af31de5aee1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
Aug 06 09:28:17 dist-test-slave-45dx krb5kdc[20947](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008497, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.113.62@KRBTEST.COM
I20260806 09:28:17.964574 21068 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1f6faea2cc554242a8bb0af31de5aee1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 09:28:17.964775 21068 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1f6faea2cc554242a8bb0af31de5aee1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:28:17.967717 21068 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1f6faea2cc554242a8bb0af31de5aee1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f6faea2cc554242a8bb0af31de5aee1" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 41769 } }
I20260806 09:28:17.968384 21068 leader_election.cc:302] T 00000000000000000000000000000000 P 1f6faea2cc554242a8bb0af31de5aee1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1f6faea2cc554242a8bb0af31de5aee1; no voters:
I20260806 09:28:17.969129 21068 leader_election.cc:288] T 00000000000000000000000000000000 P 1f6faea2cc554242a8bb0af31de5aee1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 09:28:17.969316 21082 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1f6faea2cc554242a8bb0af31de5aee1 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:28:17.970029 21082 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1f6faea2cc554242a8bb0af31de5aee1 [term 1 LEADER]: Becoming Leader. State: Replica: 1f6faea2cc554242a8bb0af31de5aee1, State: Running, Role: LEADER
I20260806 09:28:17.970669 21082 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1f6faea2cc554242a8bb0af31de5aee1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f6faea2cc554242a8bb0af31de5aee1" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 41769 } }
I20260806 09:28:17.972492 21082 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f6faea2cc554242a8bb0af31de5aee1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1f6faea2cc554242a8bb0af31de5aee1. Latest consensus state: current_term: 1 leader_uuid: "1f6faea2cc554242a8bb0af31de5aee1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f6faea2cc554242a8bb0af31de5aee1" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 41769 } } }
I20260806 09:28:17.972872 21082 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f6faea2cc554242a8bb0af31de5aee1 [sys.catalog]: This master's current role is: LEADER
I20260806 09:28:17.977875 21068 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1f6faea2cc554242a8bb0af31de5aee1 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 09:28:17.978139 21070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:17.939323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47594 (local address 127.20.113.62:41769)
0806 09:28:17.940171 (+ 848us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:17.940182 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:17.941448 (+ 1266us) server_negotiation.cc:407] Connection header received
0806 09:28:17.950982 (+ 9534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:17.951002 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:17.951382 (+ 380us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:17.951804 (+ 422us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:28:17.955234 (+ 3430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:17.956742 (+ 1508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:17.958617 (+ 1875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:17.959311 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:17.959549 (+ 238us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:17.966162 (+ 6613us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:28:17.966172 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:28:17.966188 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:28:17.966228 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:28:17.970366 (+ 4138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:17.971605 (+ 1239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:17.971614 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:17.971620 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:17.971865 (+ 245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:17.972624 (+ 759us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:17.972632 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:17.972638 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:17.973752 (+ 1114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:28:17.974215 (+ 463us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:17.975791 (+ 1576us) server_negotiation.cc:299] Negotiation successful
0806 09:28:17.976346 (+ 555us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":626,"thread_start_us":401,"threads_started":1}
I20260806 09:28:17.980515 21087 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 09:28:17.981276 21085 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f6faea2cc554242a8bb0af31de5aee1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1f6faea2cc554242a8bb0af31de5aee1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f6faea2cc554242a8bb0af31de5aee1" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 41769 } } }
I20260806 09:28:17.981722 21085 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f6faea2cc554242a8bb0af31de5aee1 [sys.catalog]: This master's current role is: LEADER
I20260806 09:28:17.986591 21087 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 09:28:17.997946 21087 catalog_manager.cc:1347] Generated new cluster ID: 06c107513d3c43958f5e5359ad86afaa
I20260806 09:28:17.998077 21087 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 09:28:18.014734 21087 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 09:28:18.016021 21087 catalog_manager.cc:1504] Loading token signing keys...
I20260806 09:28:18.026526 21087 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1f6faea2cc554242a8bb0af31de5aee1: Generated new TSK 0
I20260806 09:28:18.027324 21087 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.113.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.113.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.113.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.1.keytab.
Entry for principal kudu/127.20.113.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.1.keytab.
I20260806 09:28:18.087373 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786008495763054-20932-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786008495763054-20932-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786008495763054-20932-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786008495763054-20932-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.113.1:0
--local_ip_for_outbound_sockets=127.20.113.1
--webserver_interface=127.20.113.1
--webserver_port=0
--tserver_master_addrs=127.20.113.62:41769
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.1.keytab
--principal=kudu/127.20.113.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:40377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:28:18.451535 21115 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:28:18.451817 21115 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:28:18.452050 21115 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:28:18.459450 21115 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:28:18.459548 21115 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:28:18.459781 21115 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.113.1
I20260806 09:28:18.469548 21115 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:40377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786008495763054-20932-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786008495763054-20932-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.1.keytab
--principal=kudu/127.20.113.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.113.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786008495763054-20932-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.113.1
--webserver_port=0
--tserver_master_addrs=127.20.113.62:41769
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.113.1
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786008495763054-20932-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:28:18.470779 21115 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:28:18.472095 21115 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:28:18.486002 21121 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:28:18.485975 21124 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:28:18.486219 21126 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:28:18.486925 21115 server_base.cc:1034] running on GCE node
I20260806 09:28:18.487722 21115 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:28:18.488746 21115 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:28:18.490000 21115 hybrid_clock.cc:648] HybridClock initialized: now 1786008498489942 us; error 47 us; skew 500 ppm
I20260806 09:28:18.490311 21115 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:28:18 dist-test-slave-45dx krb5kdc[20947](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008498, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:28:18.493438 21115 init.cc:377] Logged in from keytab as kudu/127.20.113.1@KRBTEST.COM (short username kudu)
I20260806 09:28:18.495033 21115 webserver.cc:460] Webserver started at http://127.20.113.1:35777/ using document root <none> and password file <none>
I20260806 09:28:18.495692 21115 fs_manager.cc:359] Metadata directory not provided
I20260806 09:28:18.495846 21115 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:28:18.496158 21115 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:28:18.499137 21115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786008495763054-20932-0/minicluster-data/ts-0/data/instance:
uuid: "ab991acef74848e08744dee08bbb0d37"
format_stamp: "Formatted at 2026-08-06 09:28:18 on dist-test-slave-45dx"
server_key: "b91ab1efd2a07fda47e270bd92d9a058"
server_key_iv: "bdde5711a65726fbbab4d7a34618ee36"
server_key_version: "encryptionkey@0"
I20260806 09:28:18.499917 21115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786008495763054-20932-0/minicluster-data/ts-0/wal/instance:
uuid: "ab991acef74848e08744dee08bbb0d37"
format_stamp: "Formatted at 2026-08-06 09:28:18 on dist-test-slave-45dx"
server_key: "b91ab1efd2a07fda47e270bd92d9a058"
server_key_iv: "bdde5711a65726fbbab4d7a34618ee36"
server_key_version: "encryptionkey@0"
I20260806 09:28:18.505100 21115 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260806 09:28:18.508703 21138 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:28:18.509989 21115 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 09:28:18.510167 21115 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786008495763054-20932-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786008495763054-20932-0/minicluster-data/ts-0/wal
uuid: "ab991acef74848e08744dee08bbb0d37"
format_stamp: "Formatted at 2026-08-06 09:28:18 on dist-test-slave-45dx"
server_key: "b91ab1efd2a07fda47e270bd92d9a058"
server_key_iv: "bdde5711a65726fbbab4d7a34618ee36"
server_key_version: "encryptionkey@0"
I20260806 09:28:18.510399 21115 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786008495763054-20932-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786008495763054-20932-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786008495763054-20932-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:28:18.526314 21115 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:28:18.530625 21115 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:28:18.530943 21115 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:28:18.531883 21115 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:28:18.533275 21115 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:28:18.533378 21115 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:28:18.533536 21115 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:28:18.533609 21115 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:28:18.570936 21115 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.1:46451
I20260806 09:28:18.570984 21295 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.1:46451 every 8 connection(s)
I20260806 09:28:18.572278 21115 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786008495763054-20932-0/minicluster-data/ts-0/data/info.pb
I20260806 09:28:18.573102 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 21115
I20260806 09:28:18.573371 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786008495763054-20932-0/minicluster-data/ts-0/wal/instance
I20260806 09:28:18.574216 20932 external_mini_cluster.cc:1428] Setting key 93309bc5f88a55f06dc85a97b8f38a72
Aug 06 09:28:18 dist-test-slave-45dx krb5kdc[20947](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008498, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.1@KRBTEST.COM for kudu/127.20.113.62@KRBTEST.COM
I20260806 09:28:18.594821 21299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:18.575312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.1:33107 (local address 127.20.113.62:41769)
0806 09:28:18.575807 (+ 495us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:18.575811 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:18.577288 (+ 1477us) server_negotiation.cc:407] Connection header received
0806 09:28:18.579382 (+ 2094us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:18.579388 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:18.579489 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:18.579692 (+ 203us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:28:18.581241 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:18.581997 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:18.583637 (+ 1640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:18.584030 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:18.584239 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:18.588655 (+ 4416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:28:18.588660 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:28:18.588664 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:28:18.588696 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:28:18.591493 (+ 2797us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:18.592249 (+ 756us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:18.592254 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:18.592256 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:18.592435 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:18.592903 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:18.592909 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:18.592912 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:18.593188 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:28:18.593349 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:18.594177 (+ 828us) server_negotiation.cc:299] Negotiation successful
0806 09:28:18.594414 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":382,"thread_start_us":270,"threads_started":1}
I20260806 09:28:18.595916 21301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:18.575926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:41769 (local address 127.20.113.1:33107)
0806 09:28:18.577097 (+ 1171us) negotiation.cc:107] Waiting for socket to connect
0806 09:28:18.577147 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 09:28:18.579113 (+ 1966us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:28:18.579935 (+ 822us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:28:18.579943 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:28:18.580518 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:28:18.581074 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:18.581093 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:18.582171 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:18.582175 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:18.583486 (+ 1311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:18.583498 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:18.584385 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:18.584393 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:18.584615 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:18.585376 (+ 761us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:28:18.585409 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:28:18.588436 (+ 3027us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:28:18.591643 (+ 3207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:28:18.591652 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:28:18.591669 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:28:18.592109 (+ 440us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:28:18.592551 (+ 442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:28:18.592557 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:28:18.592560 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:28:18.592779 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:28:18.593329 (+ 550us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:28:18.593337 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:28:18.593796 (+ 459us) client_negotiation.cc:769] Sending connection context
0806 09:28:18.594061 (+ 265us) client_negotiation.cc:240] Negotiation successful
0806 09:28:18.594611 (+ 550us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":849,"thread_start_us":570,"threads_started":1}
WARNING: no policy specified for kudu/127.20.113.2@KRBTEST.COM; defaulting to no policy
I20260806 09:28:18.597716 21296 heartbeater.cc:347] Connected to a master server at 127.20.113.62:41769
I20260806 09:28:18.598029 21296 heartbeater.cc:464] Registering TS with master...
I20260806 09:28:18.598829 21296 heartbeater.cc:511] Master 127.20.113.62:41769 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.113.2@KRBTEST.COM" created.
I20260806 09:28:18.601096 21016 ts_manager.cc:194] Registered new tserver with Master: ab991acef74848e08744dee08bbb0d37 (127.20.113.1:46451)
I20260806 09:28:18.602576 21016 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.113.1@KRBTEST.COM'} at 127.20.113.1:33107
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.113.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.2.keytab.
Entry for principal kudu/127.20.113.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.2.keytab.
I20260806 09:28:18.625763 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786008495763054-20932-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786008495763054-20932-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786008495763054-20932-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786008495763054-20932-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.113.2:0
--local_ip_for_outbound_sockets=127.20.113.2
--webserver_interface=127.20.113.2
--webserver_port=0
--tserver_master_addrs=127.20.113.62:41769
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.2.keytab
--principal=kudu/127.20.113.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:40377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:28:18.975634 21306 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:28:18.975931 21306 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:28:18.976055 21306 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:28:18.983843 21306 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:28:18.984014 21306 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:28:18.984205 21306 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.113.2
I20260806 09:28:18.994374 21306 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:40377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786008495763054-20932-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786008495763054-20932-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.2.keytab
--principal=kudu/127.20.113.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.113.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786008495763054-20932-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.113.2
--webserver_port=0
--tserver_master_addrs=127.20.113.62:41769
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.113.2
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786008495763054-20932-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:28:18.995664 21306 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:28:18.997171 21306 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:28:19.011089 21311 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:28:19.011466 21312 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:28:19.011780 21319 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:28:19.012686 21306 server_base.cc:1034] running on GCE node
I20260806 09:28:19.013294 21306 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:28:19.014308 21306 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:28:19.015548 21306 hybrid_clock.cc:648] HybridClock initialized: now 1786008499015487 us; error 52 us; skew 500 ppm
I20260806 09:28:19.015877 21306 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:28:19 dist-test-slave-45dx krb5kdc[20947](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008499, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:28:19.019176 21306 init.cc:377] Logged in from keytab as kudu/127.20.113.2@KRBTEST.COM (short username kudu)
I20260806 09:28:19.020853 21306 webserver.cc:460] Webserver started at http://127.20.113.2:34311/ using document root <none> and password file <none>
I20260806 09:28:19.021548 21306 fs_manager.cc:359] Metadata directory not provided
I20260806 09:28:19.021701 21306 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:28:19.022061 21306 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:28:19.025121 21306 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786008495763054-20932-0/minicluster-data/ts-1/data/instance:
uuid: "3f528cbc26a8419985132e01ffaa6df5"
format_stamp: "Formatted at 2026-08-06 09:28:19 on dist-test-slave-45dx"
server_key: "f8d01cdc6e3a1769990a1e0bbd336f8c"
server_key_iv: "35787be7b1e22ad96e589deed9fc219f"
server_key_version: "encryptionkey@0"
I20260806 09:28:19.025923 21306 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786008495763054-20932-0/minicluster-data/ts-1/wal/instance:
uuid: "3f528cbc26a8419985132e01ffaa6df5"
format_stamp: "Formatted at 2026-08-06 09:28:19 on dist-test-slave-45dx"
server_key: "f8d01cdc6e3a1769990a1e0bbd336f8c"
server_key_iv: "35787be7b1e22ad96e589deed9fc219f"
server_key_version: "encryptionkey@0"
I20260806 09:28:19.031364 21306 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 09:28:19.035420 21329 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:28:19.036732 21306 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 09:28:19.036912 21306 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786008495763054-20932-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786008495763054-20932-0/minicluster-data/ts-1/wal
uuid: "3f528cbc26a8419985132e01ffaa6df5"
format_stamp: "Formatted at 2026-08-06 09:28:19 on dist-test-slave-45dx"
server_key: "f8d01cdc6e3a1769990a1e0bbd336f8c"
server_key_iv: "35787be7b1e22ad96e589deed9fc219f"
server_key_version: "encryptionkey@0"
I20260806 09:28:19.037146 21306 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786008495763054-20932-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786008495763054-20932-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786008495763054-20932-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:28:19.056085 21306 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:28:19.059988 21306 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:28:19.060285 21306 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:28:19.061455 21306 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:28:19.063006 21306 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:28:19.063117 21306 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:28:19.063275 21306 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:28:19.063352 21306 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:28:19.100947 21306 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.2:42039
I20260806 09:28:19.101054 21496 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.2:42039 every 8 connection(s)
I20260806 09:28:19.102257 21306 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786008495763054-20932-0/minicluster-data/ts-1/data/info.pb
I20260806 09:28:19.105943 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 21306
I20260806 09:28:19.106158 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786008495763054-20932-0/minicluster-data/ts-1/wal/instance
I20260806 09:28:19.106626 20932 external_mini_cluster.cc:1428] Setting key d2fa36f644103d43b3203421971945a6
Aug 06 09:28:19 dist-test-slave-45dx krb5kdc[20947](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008499, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.2@KRBTEST.COM for kudu/127.20.113.62@KRBTEST.COM
I20260806 09:28:19.124701 21501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:19.104879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.2:55719 (local address 127.20.113.62:41769)
0806 09:28:19.105316 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:19.105321 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:19.106640 (+ 1319us) server_negotiation.cc:407] Connection header received
0806 09:28:19.109182 (+ 2542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:19.109187 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:19.109268 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:19.109403 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:28:19.110965 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:19.112224 (+ 1259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:19.113805 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:19.114574 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:19.114715 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:19.118854 (+ 4139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:28:19.118859 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:28:19.118862 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:28:19.118894 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:28:19.121638 (+ 2744us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:19.122395 (+ 757us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:19.122404 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:19.122407 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:19.122533 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:19.122983 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:19.122987 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:19.122989 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:19.123231 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:28:19.123388 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:19.124059 (+ 671us) server_negotiation.cc:299] Negotiation successful
0806 09:28:19.124270 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":213,"threads_started":1}
I20260806 09:28:19.125638 21502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:19.105500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:41769 (local address 127.20.113.2:55719)
0806 09:28:19.106323 (+ 823us) negotiation.cc:107] Waiting for socket to connect
0806 09:28:19.106377 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 09:28:19.108850 (+ 2473us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:28:19.109599 (+ 749us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:28:19.109607 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:28:19.110113 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:28:19.110770 (+ 657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:19.110790 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:19.112434 (+ 1644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:19.112437 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:19.113614 (+ 1177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:19.113626 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:19.114774 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:19.114779 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:19.115147 (+ 368us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:19.115935 (+ 788us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:28:19.115957 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:28:19.118630 (+ 2673us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:28:19.121814 (+ 3184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:28:19.121823 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:28:19.121839 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:28:19.122256 (+ 417us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:28:19.122648 (+ 392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:28:19.122653 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:28:19.122657 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:28:19.122847 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:28:19.123378 (+ 531us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:28:19.123386 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:28:19.123693 (+ 307us) client_negotiation.cc:769] Sending connection context
0806 09:28:19.123957 (+ 264us) client_negotiation.cc:240] Negotiation successful
0806 09:28:19.124510 (+ 553us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":571,"thread_start_us":350,"threads_started":1}
I20260806 09:28:19.127161 21498 heartbeater.cc:347] Connected to a master server at 127.20.113.62:41769
I20260806 09:28:19.127473 21498 heartbeater.cc:464] Registering TS with master...
I20260806 09:28:19.128185 21498 heartbeater.cc:511] Master 127.20.113.62:41769 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.20.113.3@KRBTEST.COM; defaulting to no policy
I20260806 09:28:19.129662 21016 ts_manager.cc:194] Registered new tserver with Master: 3f528cbc26a8419985132e01ffaa6df5 (127.20.113.2:42039)
I20260806 09:28:19.130492 21016 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.113.2@KRBTEST.COM'} at 127.20.113.2:55719
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.113.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.113.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.3.keytab.
Entry for principal kudu/127.20.113.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.3.keytab.
I20260806 09:28:19.155256 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786008495763054-20932-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786008495763054-20932-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786008495763054-20932-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786008495763054-20932-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.113.3:0
--local_ip_for_outbound_sockets=127.20.113.3
--webserver_interface=127.20.113.3
--webserver_port=0
--tserver_master_addrs=127.20.113.62:41769
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.3.keytab
--principal=kudu/127.20.113.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:40377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:28:19.500248 21505 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:28:19.500612 21505 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:28:19.500727 21505 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:28:19.508199 21505 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:28:19.508375 21505 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:28:19.508503 21505 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.113.3
I20260806 09:28:19.518647 21505 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:40377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786008495763054-20932-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786008495763054-20932-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.3.keytab
--principal=kudu/127.20.113.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.113.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786008495763054-20932-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.113.3
--webserver_port=0
--tserver_master_addrs=127.20.113.62:41769
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.113.3
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786008495763054-20932-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:28:19.519935 21505 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:28:19.521335 21505 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:28:19.535094 21513 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:28:19.535167 21518 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:28:19.535864 21514 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:28:19.536289 21505 server_base.cc:1034] running on GCE node
I20260806 09:28:19.536995 21505 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:28:19.537910 21505 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:28:19.539160 21505 hybrid_clock.cc:648] HybridClock initialized: now 1786008499539076 us; error 52 us; skew 500 ppm
I20260806 09:28:19.539539 21505 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:28:19 dist-test-slave-45dx krb5kdc[20947](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008499, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:28:19.543049 21505 init.cc:377] Logged in from keytab as kudu/127.20.113.3@KRBTEST.COM (short username kudu)
I20260806 09:28:19.544523 21505 webserver.cc:460] Webserver started at http://127.20.113.3:46075/ using document root <none> and password file <none>
I20260806 09:28:19.545214 21505 fs_manager.cc:359] Metadata directory not provided
I20260806 09:28:19.545403 21505 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:28:19.545778 21505 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:28:19.548816 21505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786008495763054-20932-0/minicluster-data/ts-2/data/instance:
uuid: "dceccb38fb874fa399fb7f3d2dd64a30"
format_stamp: "Formatted at 2026-08-06 09:28:19 on dist-test-slave-45dx"
server_key: "85082d81a7704de8a8f1ba39c81bc323"
server_key_iv: "d3fb3e2b2b07c49060b234e4ebd039d2"
server_key_version: "encryptionkey@0"
I20260806 09:28:19.549620 21505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786008495763054-20932-0/minicluster-data/ts-2/wal/instance:
uuid: "dceccb38fb874fa399fb7f3d2dd64a30"
format_stamp: "Formatted at 2026-08-06 09:28:19 on dist-test-slave-45dx"
server_key: "85082d81a7704de8a8f1ba39c81bc323"
server_key_iv: "d3fb3e2b2b07c49060b234e4ebd039d2"
server_key_version: "encryptionkey@0"
I20260806 09:28:19.554952 21505 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260806 09:28:19.558552 21528 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:28:19.559801 21505 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 09:28:19.559986 21505 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786008495763054-20932-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786008495763054-20932-0/minicluster-data/ts-2/wal
uuid: "dceccb38fb874fa399fb7f3d2dd64a30"
format_stamp: "Formatted at 2026-08-06 09:28:19 on dist-test-slave-45dx"
server_key: "85082d81a7704de8a8f1ba39c81bc323"
server_key_iv: "d3fb3e2b2b07c49060b234e4ebd039d2"
server_key_version: "encryptionkey@0"
I20260806 09:28:19.560192 21505 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786008495763054-20932-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786008495763054-20932-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786008495763054-20932-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:28:19.574108 21505 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:28:19.578558 21505 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:28:19.578845 21505 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:28:19.579854 21505 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:28:19.581265 21505 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:28:19.581379 21505 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:28:19.581526 21505 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:28:19.581605 21505 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:28:19.605722 21296 heartbeater.cc:503] Master 127.20.113.62:41769 was elected leader, sending a full tablet report...
I20260806 09:28:19.624413 21505 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.3:34893
I20260806 09:28:19.624500 21695 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.3:34893 every 8 connection(s)
I20260806 09:28:19.625839 21505 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786008495763054-20932-0/minicluster-data/ts-2/data/info.pb
I20260806 09:28:19.634511 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 21505
I20260806 09:28:19.634771 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786008495763054-20932-0/minicluster-data/ts-2/wal/instance
I20260806 09:28:19.635251 20932 external_mini_cluster.cc:1428] Setting key af2207ab8d5a67c282db9013e231e909
Aug 06 09:28:19 dist-test-slave-45dx krb5kdc[20947](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008499, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.3@KRBTEST.COM for kudu/127.20.113.62@KRBTEST.COM
I20260806 09:28:19.647994 21698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:19.628848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.3:42865 (local address 127.20.113.62:41769)
0806 09:28:19.629411 (+ 563us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:19.629416 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:19.630423 (+ 1007us) server_negotiation.cc:407] Connection header received
0806 09:28:19.632531 (+ 2108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:19.632537 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:19.632660 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:19.632792 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:28:19.634753 (+ 1961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:19.635756 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:19.637750 (+ 1994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:19.638399 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:19.638546 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:19.642594 (+ 4048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:28:19.642599 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:28:19.642602 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:28:19.642636 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:28:19.645297 (+ 2661us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:19.645995 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:19.645999 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:19.646002 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:19.646120 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:19.646524 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:19.646528 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:19.646529 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:19.646778 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:28:19.646903 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:19.647559 (+ 656us) server_negotiation.cc:299] Negotiation successful
0806 09:28:19.647717 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":430,"thread_start_us":315,"threads_started":1}
I20260806 09:28:19.649108 21699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:19.629416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:41769 (local address 127.20.113.3:42865)
0806 09:28:19.630235 (+ 819us) negotiation.cc:107] Waiting for socket to connect
0806 09:28:19.630277 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 09:28:19.632126 (+ 1849us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:28:19.633085 (+ 959us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:28:19.633095 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:28:19.633665 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:28:19.634482 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:19.634513 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:19.635977 (+ 1464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:19.635983 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:19.637580 (+ 1597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:19.637599 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:19.638617 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:19.638622 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:19.638851 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:19.639662 (+ 811us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:28:19.639687 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:28:19.642386 (+ 2699us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:28:19.645430 (+ 3044us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:28:19.645437 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:28:19.645449 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:28:19.645884 (+ 435us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:28:19.646215 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:28:19.646218 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:28:19.646221 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:28:19.646424 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:28:19.646905 (+ 481us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:28:19.646912 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:28:19.647216 (+ 304us) client_negotiation.cc:769] Sending connection context
0806 09:28:19.647455 (+ 239us) client_negotiation.cc:240] Negotiation successful
0806 09:28:19.647850 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":561,"thread_start_us":340,"threads_started":1}
I20260806 09:28:19.650619 21696 heartbeater.cc:347] Connected to a master server at 127.20.113.62:41769
I20260806 09:28:19.650988 21696 heartbeater.cc:464] Registering TS with master...
I20260806 09:28:19.651994 21696 heartbeater.cc:511] Master 127.20.113.62:41769 requested a full tablet report, sending...
I20260806 09:28:19.653445 21016 ts_manager.cc:194] Registered new tserver with Master: dceccb38fb874fa399fb7f3d2dd64a30 (127.20.113.3:34893)
I20260806 09:28:19.654234 21016 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.113.3@KRBTEST.COM'} at 127.20.113.3:42865
I20260806 09:28:19.663823 20932 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 09:28:19 dist-test-slave-45dx krb5kdc[20947](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008499, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 09:28:19 dist-test-slave-45dx krb5kdc[20947](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008499, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.20.113.1@KRBTEST.COM
I20260806 09:28:19.721513 21706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:19.705958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40598 (local address 127.20.113.1:46451)
0806 09:28:19.706619 (+ 661us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:19.706626 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:19.706656 (+ 30us) server_negotiation.cc:407] Connection header received
0806 09:28:19.706722 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:19.706729 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:19.706956 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:19.707092 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:28:19.708706 (+ 1614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:19.709776 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:19.711051 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:19.711624 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:19.711721 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:19.715995 (+ 4274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:28:19.716007 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:28:19.716022 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:28:19.716058 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:28:19.718882 (+ 2824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:19.719492 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:19.719499 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:19.719501 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:19.719636 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:19.719988 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:19.719991 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:19.719993 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:19.720456 (+ 463us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:28:19.720698 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:19.720930 (+ 232us) server_negotiation.cc:299] Negotiation successful
0806 09:28:19.721148 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":503,"thread_start_us":308,"threads_started":1}
W20260806 09:28:19.722869 21175 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40598
Aug 06 09:28:19 dist-test-slave-45dx krb5kdc[20947](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008499, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 09:28:19 dist-test-slave-45dx krb5kdc[20947](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008499, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.113.1@KRBTEST.COM
I20260806 09:28:19.774861 21706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:19.761019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40600 (local address 127.20.113.1:46451)
0806 09:28:19.761261 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:19.761269 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:19.761555 (+ 286us) server_negotiation.cc:407] Connection header received
0806 09:28:19.761781 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:19.761785 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:19.761857 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:19.761961 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:28:19.763319 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:19.764025 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:19.765320 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:19.765743 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:19.765885 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:19.769694 (+ 3809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:28:19.769699 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:28:19.769702 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:28:19.769737 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:28:19.772277 (+ 2540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:19.772961 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:19.772967 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:19.772970 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:19.773102 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:19.773472 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:19.773477 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:19.773480 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:19.773742 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:28:19.774002 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:19.774320 (+ 318us) server_negotiation.cc:299] Negotiation successful
0806 09:28:19.774540 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260806 09:28:19.776387 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 21115
I20260806 09:28:19.788694 21706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:19.777283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40606 (local address 127.20.113.1:46451)
0806 09:28:19.777548 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:19.777554 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:19.777804 (+ 250us) server_negotiation.cc:407] Connection header received
0806 09:28:19.777913 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:19.777917 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:19.777972 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:19.778047 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:28:19.779240 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:19.779921 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:19.781186 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:19.781605 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:19.781693 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:19.783847 (+ 2154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:28:19.783852 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:28:19.783855 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:28:19.783883 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:28:19.786424 (+ 2541us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:19.786989 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:19.786992 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:19.786994 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:19.787119 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:19.787472 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:19.787475 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:19.787477 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:19.787706 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:28:19.787879 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:19.788113 (+ 234us) server_negotiation.cc:299] Negotiation successful
0806 09:28:19.788257 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 09:28:19.789366 21289 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:28:19.874080 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 21115
I20260806 09:28:19.899672 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 21306
Aug 06 09:28:19 dist-test-slave-45dx krb5kdc[20947](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008499, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.113.2@KRBTEST.COM
I20260806 09:28:19.915592 21719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:19.900628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58428 (local address 127.20.113.2:42039)
0806 09:28:19.901159 (+ 531us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:19.901166 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:19.901192 (+ 26us) server_negotiation.cc:407] Connection header received
0806 09:28:19.901254 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:19.901260 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:19.901474 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:19.901653 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:28:19.902957 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:19.903971 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:19.905413 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:19.905969 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:19.906107 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:19.910210 (+ 4103us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:28:19.910222 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:28:19.910237 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:28:19.910277 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:28:19.913067 (+ 2790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:19.913686 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:19.913693 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:19.913695 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:19.913828 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:19.914171 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:19.914174 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:19.914176 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:19.914506 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:28:19.914730 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:19.915066 (+ 336us) server_negotiation.cc:299] Negotiation successful
0806 09:28:19.915277 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":253,"threads_started":1}
I20260806 09:28:19.916978 21489 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:28:20.005856 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 21306
I20260806 09:28:20.029877 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 21505
Aug 06 09:28:20 dist-test-slave-45dx krb5kdc[20947](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008499, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.113.3@KRBTEST.COM
I20260806 09:28:20.046413 21723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:20.031010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40646 (local address 127.20.113.3:34893)
0806 09:28:20.031648 (+ 638us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:20.031656 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:20.031685 (+ 29us) server_negotiation.cc:407] Connection header received
0806 09:28:20.031747 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:20.031753 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:20.031965 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:20.032096 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:28:20.033403 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:20.034526 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:20.035732 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:20.036266 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:20.036458 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:20.040743 (+ 4285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:28:20.040755 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:28:20.040770 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:28:20.040803 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:28:20.043614 (+ 2811us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:20.044533 (+ 919us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:20.044550 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:20.044554 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:20.044695 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:20.045029 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:20.045032 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:20.045034 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:20.045401 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:28:20.045652 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:20.045910 (+ 258us) server_negotiation.cc:299] Negotiation successful
0806 09:28:20.046110 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":494,"thread_start_us":326,"threads_started":1}
I20260806 09:28:20.047749 21689 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:28:20.131014 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 21505
I20260806 09:28:20.157310 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 20965
I20260806 09:28:20.158459 21063 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:28:20.264770 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 20965
2026-08-06T09:28:20Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (4462 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786008495763054-20932-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 09:28:20 dist-test-slave-45dx krb5kdc[21733](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 09:28:20 dist-test-slave-45dx krb5kdc[21733](info): set up 2 sockets
Aug 06 09:28:20 dist-test-slave-45dx krb5kdc[21733](info): commencing operation
krb5kdc: starting...
W20260806 09:28:21.534083 20932 mini_kdc.cc:121] Time spent starting KDC: real 1.217s user 0.001s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786008495763054-20932-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786008495763054-20932-0/krb5kdc/test-user.keytab.
Aug 06 09:28:21 dist-test-slave-45dx krb5kdc[21733](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008501, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T09:28:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T09:28:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.113.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.113.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.113.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab.
Entry for principal kudu/127.20.113.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab.
I20260806 09:28:21.747648 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786008495763054-20932-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786008495763054-20932-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786008495763054-20932-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.113.62:33629
--webserver_interface=127.20.113.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab
--principal=kudu/127.20.113.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:41193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.113.62:33629
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:28:22.085171 21749 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:28:22.085525 21749 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:28:22.085677 21749 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:28:22.092850 21749 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 09:28:22.092999 21749 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:28:22.093107 21749 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:28:22.093217 21749 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 09:28:22.093291 21749 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 09:28:22.103204 21749 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:41193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786008495763054-20932-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786008495763054-20932-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.113.62:33629
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab
--principal=kudu/127.20.113.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.113.62:33629
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.113.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786008495763054-20932-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:28:22.104614 21749 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:28:22.105971 21749 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:28:22.118171 21758 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:28:22.118237 21764 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:28:22.118289 21762 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:28:22.118582 21749 server_base.cc:1034] running on GCE node
I20260806 09:28:22.119573 21749 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:28:22.121002 21749 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:28:22.122232 21749 hybrid_clock.cc:648] HybridClock initialized: now 1786008502122181 us; error 42 us; skew 500 ppm
I20260806 09:28:22.122577 21749 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:28:22 dist-test-slave-45dx krb5kdc[21733](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008502, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:28:22.130498 21749 init.cc:377] Logged in from keytab as kudu/127.20.113.62@KRBTEST.COM (short username kudu)
I20260806 09:28:22.132052 21749 webserver.cc:460] Webserver started at http://127.20.113.62:41347/ using document root <none> and password file <none>
I20260806 09:28:22.132844 21749 fs_manager.cc:359] Metadata directory not provided
I20260806 09:28:22.132992 21749 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:28:22.133319 21749 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:28:22.136418 21749 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786008495763054-20932-0/minicluster-data/master-0/data/instance:
uuid: "84552df203414027849a83685dd0e9ba"
format_stamp: "Formatted at 2026-08-06 09:28:22 on dist-test-slave-45dx"
server_key: "258a6eadd920dcf7dd7ddd9349fe23ab"
server_key_iv: "883b03c395694aaf2b1251e8109d1904"
server_key_version: "encryptionkey@0"
I20260806 09:28:22.137179 21749 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786008495763054-20932-0/minicluster-data/master-0/wal/instance:
uuid: "84552df203414027849a83685dd0e9ba"
format_stamp: "Formatted at 2026-08-06 09:28:22 on dist-test-slave-45dx"
server_key: "258a6eadd920dcf7dd7ddd9349fe23ab"
server_key_iv: "883b03c395694aaf2b1251e8109d1904"
server_key_version: "encryptionkey@0"
I20260806 09:28:22.142477 21749 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260806 09:28:22.146044 21775 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:28:22.147413 21749 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 09:28:22.147585 21749 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786008495763054-20932-0/minicluster-data/master-0/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786008495763054-20932-0/minicluster-data/master-0/wal
uuid: "84552df203414027849a83685dd0e9ba"
format_stamp: "Formatted at 2026-08-06 09:28:22 on dist-test-slave-45dx"
server_key: "258a6eadd920dcf7dd7ddd9349fe23ab"
server_key_iv: "883b03c395694aaf2b1251e8109d1904"
server_key_version: "encryptionkey@0"
I20260806 09:28:22.147785 21749 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786008495763054-20932-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786008495763054-20932-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786008495763054-20932-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:28:22.171074 21749 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:28:22.175199 21749 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:28:22.175503 21749 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:28:22.194556 21749 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.62:33629
I20260806 09:28:22.194577 21842 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.62:33629 every 8 connection(s)
I20260806 09:28:22.196027 21749 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
I20260806 09:28:22.199826 21843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:28:22.206439 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 21749
I20260806 09:28:22.206720 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786008495763054-20932-0/minicluster-data/master-0/wal/instance
I20260806 09:28:22.207623 20932 external_mini_cluster.cc:1428] Setting key 0fa04487f30af6ddf757f7b963d40981
I20260806 09:28:22.211437 21843 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84552df203414027849a83685dd0e9ba: Bootstrap starting.
I20260806 09:28:22.215518 21843 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 84552df203414027849a83685dd0e9ba: Neither blocks nor log segments found. Creating new log.
I20260806 09:28:22.216678 21843 log.cc:824] T 00000000000000000000000000000000 P 84552df203414027849a83685dd0e9ba: Log is configured to *not* fsync() on all Append() calls
I20260806 09:28:22.219580 21843 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84552df203414027849a83685dd0e9ba: No bootstrap required, opened a new log
Aug 06 09:28:22 dist-test-slave-45dx krb5kdc[21733](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008501, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.113.62@KRBTEST.COM
I20260806 09:28:22.227442 21843 raft_consensus.cc:348] T 00000000000000000000000000000000 P 84552df203414027849a83685dd0e9ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84552df203414027849a83685dd0e9ba" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 33629 } }
I20260806 09:28:22.227818 21843 raft_consensus.cc:374] T 00000000000000000000000000000000 P 84552df203414027849a83685dd0e9ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:28:22.227972 21843 raft_consensus.cc:729] T 00000000000000000000000000000000 P 84552df203414027849a83685dd0e9ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84552df203414027849a83685dd0e9ba, State: Initialized, Role: FOLLOWER
I20260806 09:28:22.228614 21843 consensus_queue.cc:260] T 00000000000000000000000000000000 P 84552df203414027849a83685dd0e9ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84552df203414027849a83685dd0e9ba" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 33629 } }
I20260806 09:28:22.228890 21843 raft_consensus.cc:388] T 00000000000000000000000000000000 P 84552df203414027849a83685dd0e9ba [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 09:28:22.229081 21843 raft_consensus.cc:482] T 00000000000000000000000000000000 P 84552df203414027849a83685dd0e9ba [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 09:28:22.229295 21843 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 84552df203414027849a83685dd0e9ba [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:28:22.231297 21847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:22.209883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49916 (local address 127.20.113.62:33629)
0806 09:28:22.210649 (+ 766us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:22.210660 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:22.210700 (+ 40us) server_negotiation.cc:407] Connection header received
0806 09:28:22.212561 (+ 1861us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:22.212585 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:22.212947 (+ 362us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:22.213267 (+ 320us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:28:22.215264 (+ 1997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:22.216735 (+ 1471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:22.218378 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:22.219055 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:22.219254 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:22.223306 (+ 4052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:28:22.223313 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:28:22.223330 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:28:22.223366 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:28:22.226570 (+ 3204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:22.227242 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:22.227254 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:22.227260 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:22.227436 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:22.227809 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:22.227816 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:22.227818 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:22.228529 (+ 711us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:28:22.228857 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:22.229384 (+ 527us) server_negotiation.cc:299] Negotiation successful
0806 09:28:22.229854 (+ 470us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":565,"thread_start_us":354,"threads_started":1}
I20260806 09:28:22.232493 21843 raft_consensus.cc:504] T 00000000000000000000000000000000 P 84552df203414027849a83685dd0e9ba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84552df203414027849a83685dd0e9ba" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 33629 } }
I20260806 09:28:22.232976 21843 leader_election.cc:302] T 00000000000000000000000000000000 P 84552df203414027849a83685dd0e9ba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 84552df203414027849a83685dd0e9ba; no voters:
I20260806 09:28:22.233649 21843 leader_election.cc:288] T 00000000000000000000000000000000 P 84552df203414027849a83685dd0e9ba [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 09:28:22.233773 21849 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 84552df203414027849a83685dd0e9ba [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:28:22.234539 21849 raft_consensus.cc:686] T 00000000000000000000000000000000 P 84552df203414027849a83685dd0e9ba [term 1 LEADER]: Becoming Leader. State: Replica: 84552df203414027849a83685dd0e9ba, State: Running, Role: LEADER
I20260806 09:28:22.235195 21849 consensus_queue.cc:237] T 00000000000000000000000000000000 P 84552df203414027849a83685dd0e9ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84552df203414027849a83685dd0e9ba" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 33629 } }
I20260806 09:28:22.236167 21843 sys_catalog.cc:564] T 00000000000000000000000000000000 P 84552df203414027849a83685dd0e9ba [sys.catalog]: configured and running, proceeding with master startup.
I20260806 09:28:22.238128 21851 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84552df203414027849a83685dd0e9ba [sys.catalog]: SysCatalogTable state changed. Reason: New leader 84552df203414027849a83685dd0e9ba. Latest consensus state: current_term: 1 leader_uuid: "84552df203414027849a83685dd0e9ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84552df203414027849a83685dd0e9ba" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 33629 } } }
I20260806 09:28:22.238591 21851 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84552df203414027849a83685dd0e9ba [sys.catalog]: This master's current role is: LEADER
I20260806 09:28:22.239146 21853 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84552df203414027849a83685dd0e9ba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "84552df203414027849a83685dd0e9ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84552df203414027849a83685dd0e9ba" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 33629 } } }
I20260806 09:28:22.239552 21853 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84552df203414027849a83685dd0e9ba [sys.catalog]: This master's current role is: LEADER
I20260806 09:28:22.246970 21856 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 09:28:22.252251 21856 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 09:28:22.260911 21856 catalog_manager.cc:1347] Generated new cluster ID: 502fb6d950044b75882f53cbc18f297b
I20260806 09:28:22.261022 21856 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 09:28:22.270277 21856 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 09:28:22.271996 21856 catalog_manager.cc:1504] Loading token signing keys...
I20260806 09:28:22.290640 21856 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 84552df203414027849a83685dd0e9ba: Generated new TSK 0
I20260806 09:28:22.291458 21856 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.113.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.113.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.113.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.1.keytab.
Entry for principal kudu/127.20.113.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.1.keytab.
I20260806 09:28:22.359150 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786008495763054-20932-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786008495763054-20932-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786008495763054-20932-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786008495763054-20932-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.113.1:0
--local_ip_for_outbound_sockets=127.20.113.1
--webserver_interface=127.20.113.1
--webserver_port=0
--tserver_master_addrs=127.20.113.62:33629
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.1.keytab
--principal=kudu/127.20.113.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:41193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:28:22.708410 21886 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:28:22.708683 21886 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:28:22.708884 21886 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:28:22.716044 21886 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:28:22.716173 21886 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:28:22.716403 21886 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.113.1
I20260806 09:28:22.726265 21886 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:41193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786008495763054-20932-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786008495763054-20932-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.1.keytab
--principal=kudu/127.20.113.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.113.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786008495763054-20932-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.113.1
--webserver_port=0
--tserver_master_addrs=127.20.113.62:33629
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.113.1
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786008495763054-20932-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:28:22.727499 21886 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:28:22.728899 21886 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:28:22.743057 21897 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:28:22.743033 21891 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:28:22.743165 21892 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:28:22.744707 21886 server_base.cc:1034] running on GCE node
I20260806 09:28:22.745458 21886 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:28:22.746414 21886 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:28:22.747690 21886 hybrid_clock.cc:648] HybridClock initialized: now 1786008502747621 us; error 40 us; skew 500 ppm
I20260806 09:28:22.748118 21886 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:28:22 dist-test-slave-45dx krb5kdc[21733](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008502, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:28:22.751322 21886 init.cc:377] Logged in from keytab as kudu/127.20.113.1@KRBTEST.COM (short username kudu)
I20260806 09:28:22.752840 21886 webserver.cc:460] Webserver started at http://127.20.113.1:42217/ using document root <none> and password file <none>
I20260806 09:28:22.753538 21886 fs_manager.cc:359] Metadata directory not provided
I20260806 09:28:22.753684 21886 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:28:22.753999 21886 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:28:22.756865 21886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786008495763054-20932-0/minicluster-data/ts-0/data/instance:
uuid: "6561d735f1f1469eabf2d624047dc889"
format_stamp: "Formatted at 2026-08-06 09:28:22 on dist-test-slave-45dx"
server_key: "3783aaaa88dffd6260758ab14d9a2e40"
server_key_iv: "829af541e15e9a1d62b410e920a349e2"
server_key_version: "encryptionkey@0"
I20260806 09:28:22.757654 21886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786008495763054-20932-0/minicluster-data/ts-0/wal/instance:
uuid: "6561d735f1f1469eabf2d624047dc889"
format_stamp: "Formatted at 2026-08-06 09:28:22 on dist-test-slave-45dx"
server_key: "3783aaaa88dffd6260758ab14d9a2e40"
server_key_iv: "829af541e15e9a1d62b410e920a349e2"
server_key_version: "encryptionkey@0"
I20260806 09:28:22.762584 21886 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 09:28:22.765934 21909 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:28:22.767282 21886 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 09:28:22.767467 21886 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786008495763054-20932-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786008495763054-20932-0/minicluster-data/ts-0/wal
uuid: "6561d735f1f1469eabf2d624047dc889"
format_stamp: "Formatted at 2026-08-06 09:28:22 on dist-test-slave-45dx"
server_key: "3783aaaa88dffd6260758ab14d9a2e40"
server_key_iv: "829af541e15e9a1d62b410e920a349e2"
server_key_version: "encryptionkey@0"
I20260806 09:28:22.767729 21886 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786008495763054-20932-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786008495763054-20932-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786008495763054-20932-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:28:22.787330 21886 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:28:22.791520 21886 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:28:22.791831 21886 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:28:22.792881 21886 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:28:22.794209 21886 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:28:22.794315 21886 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:28:22.794473 21886 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:28:22.794545 21886 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:28:22.828469 21886 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.1:38205
I20260806 09:28:22.828519 22076 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.1:38205 every 8 connection(s)
I20260806 09:28:22.829725 21886 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786008495763054-20932-0/minicluster-data/ts-0/data/info.pb
I20260806 09:28:22.833092 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 21886
I20260806 09:28:22.833338 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786008495763054-20932-0/minicluster-data/ts-0/wal/instance
I20260806 09:28:22.834040 20932 external_mini_cluster.cc:1428] Setting key 1da98080a2f5d7484a5fa09b67b0046a
Aug 06 09:28:22 dist-test-slave-45dx krb5kdc[21733](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008502, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.1@KRBTEST.COM for kudu/127.20.113.62@KRBTEST.COM
I20260806 09:28:22.852607 22081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:22.832321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.1:44639 (local address 127.20.113.62:33629)
0806 09:28:22.832893 (+ 572us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:22.832900 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:22.833899 (+ 999us) server_negotiation.cc:407] Connection header received
0806 09:28:22.835766 (+ 1867us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:22.835771 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:22.835858 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:22.835986 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:28:22.837669 (+ 1683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:22.838418 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:22.839986 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:22.840496 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:22.840632 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:22.845545 (+ 4913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:28:22.845550 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:28:22.845553 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:28:22.845585 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:28:22.849085 (+ 3500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:22.849804 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:22.849813 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:22.849819 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:22.850006 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:22.850424 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:22.850432 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:22.850438 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:22.850875 (+ 437us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:28:22.851028 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:22.851767 (+ 739us) server_negotiation.cc:299] Negotiation successful
0806 09:28:22.852077 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":403,"thread_start_us":247,"threads_started":1}
I20260806 09:28:22.853327 22083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:22.832883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:33629 (local address 127.20.113.1:44639)
0806 09:28:22.833694 (+ 811us) negotiation.cc:107] Waiting for socket to connect
0806 09:28:22.833749 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 09:28:22.835495 (+ 1746us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:28:22.836219 (+ 724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:28:22.836227 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:28:22.836872 (+ 645us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:28:22.837471 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:22.837504 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:22.838565 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:22.838570 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:22.839830 (+ 1260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:22.839842 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:22.840829 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:22.840866 (+ 37us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:22.841219 (+ 353us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:22.842092 (+ 873us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:28:22.842133 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:28:22.845321 (+ 3188us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:28:22.849223 (+ 3902us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:28:22.849229 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:28:22.849240 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:28:22.849649 (+ 409us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:28:22.850120 (+ 471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:28:22.850123 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:28:22.850125 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:28:22.850299 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:28:22.851001 (+ 702us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:28:22.851007 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:28:22.851345 (+ 338us) client_negotiation.cc:769] Sending connection context
0806 09:28:22.851558 (+ 213us) client_negotiation.cc:240] Negotiation successful
0806 09:28:22.852023 (+ 465us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556,"thread_start_us":351,"threads_started":1}
I20260806 09:28:22.855194 22078 heartbeater.cc:347] Connected to a master server at 127.20.113.62:33629
I20260806 09:28:22.855490 22078 heartbeater.cc:464] Registering TS with master...
I20260806 09:28:22.856174 22078 heartbeater.cc:511] Master 127.20.113.62:33629 requested a full tablet report, sending...
I20260806 09:28:22.858560 21795 ts_manager.cc:194] Registered new tserver with Master: 6561d735f1f1469eabf2d624047dc889 (127.20.113.1:38205)
WARNING: no policy specified for kudu/127.20.113.2@KRBTEST.COM; defaulting to no policy
I20260806 09:28:22.860194 21795 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.113.1@KRBTEST.COM'} at 127.20.113.1:44639
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.113.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.113.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.2.keytab.
Entry for principal kudu/127.20.113.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.2.keytab.
I20260806 09:28:22.887272 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786008495763054-20932-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786008495763054-20932-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786008495763054-20932-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786008495763054-20932-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.113.2:0
--local_ip_for_outbound_sockets=127.20.113.2
--webserver_interface=127.20.113.2
--webserver_port=0
--tserver_master_addrs=127.20.113.62:33629
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.2.keytab
--principal=kudu/127.20.113.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:41193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:28:23.225807 22086 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:28:23.226060 22086 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:28:23.226243 22086 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:28:23.233165 22086 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:28:23.233282 22086 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:28:23.233466 22086 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.113.2
I20260806 09:28:23.243392 22086 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:41193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786008495763054-20932-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786008495763054-20932-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.2.keytab
--principal=kudu/127.20.113.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.113.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786008495763054-20932-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.113.2
--webserver_port=0
--tserver_master_addrs=127.20.113.62:33629
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.113.2
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786008495763054-20932-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:28:23.244777 22086 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:28:23.246233 22086 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:28:23.259904 22091 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:28:23.259966 22092 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:28:23.259857 22094 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:28:23.260988 22086 server_base.cc:1034] running on GCE node
I20260806 09:28:23.261595 22086 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:28:23.262533 22086 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:28:23.263769 22086 hybrid_clock.cc:648] HybridClock initialized: now 1786008503263713 us; error 59 us; skew 500 ppm
I20260806 09:28:23.264088 22086 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:28:23 dist-test-slave-45dx krb5kdc[21733](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008503, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:28:23.267248 22086 init.cc:377] Logged in from keytab as kudu/127.20.113.2@KRBTEST.COM (short username kudu)
I20260806 09:28:23.268766 22086 webserver.cc:460] Webserver started at http://127.20.113.2:34533/ using document root <none> and password file <none>
I20260806 09:28:23.269436 22086 fs_manager.cc:359] Metadata directory not provided
I20260806 09:28:23.269582 22086 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:28:23.269912 22086 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:28:23.272910 22086 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786008495763054-20932-0/minicluster-data/ts-1/data/instance:
uuid: "c40b3247fecf4ecba4f70be898f65d7b"
format_stamp: "Formatted at 2026-08-06 09:28:23 on dist-test-slave-45dx"
server_key: "384292155c75685a2bd66fccce46bb59"
server_key_iv: "9710576184c7bbb37a89106e59e51667"
server_key_version: "encryptionkey@0"
I20260806 09:28:23.273693 22086 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786008495763054-20932-0/minicluster-data/ts-1/wal/instance:
uuid: "c40b3247fecf4ecba4f70be898f65d7b"
format_stamp: "Formatted at 2026-08-06 09:28:23 on dist-test-slave-45dx"
server_key: "384292155c75685a2bd66fccce46bb59"
server_key_iv: "9710576184c7bbb37a89106e59e51667"
server_key_version: "encryptionkey@0"
I20260806 09:28:23.278571 22086 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 09:28:23.281932 22104 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:28:23.283123 22086 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 09:28:23.283298 22086 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786008495763054-20932-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786008495763054-20932-0/minicluster-data/ts-1/wal
uuid: "c40b3247fecf4ecba4f70be898f65d7b"
format_stamp: "Formatted at 2026-08-06 09:28:23 on dist-test-slave-45dx"
server_key: "384292155c75685a2bd66fccce46bb59"
server_key_iv: "9710576184c7bbb37a89106e59e51667"
server_key_version: "encryptionkey@0"
I20260806 09:28:23.283489 22086 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786008495763054-20932-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786008495763054-20932-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786008495763054-20932-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:28:23.313573 22086 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:28:23.318315 22086 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:28:23.318631 22086 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:28:23.319682 22086 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:28:23.321141 22086 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:28:23.321242 22086 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:28:23.321388 22086 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:28:23.321465 22086 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:28:23.355242 22086 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.2:34717
I20260806 09:28:23.355289 22268 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.2:34717 every 8 connection(s)
I20260806 09:28:23.356633 22086 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786008495763054-20932-0/minicluster-data/ts-1/data/info.pb
I20260806 09:28:23.366906 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 22086
I20260806 09:28:23.367131 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786008495763054-20932-0/minicluster-data/ts-1/wal/instance
I20260806 09:28:23.367662 20932 external_mini_cluster.cc:1428] Setting key 1268b83f765f427001fc45e6e46c9173
Aug 06 09:28:23 dist-test-slave-45dx krb5kdc[21733](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008503, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.2@KRBTEST.COM for kudu/127.20.113.62@KRBTEST.COM
I20260806 09:28:23.376159 22274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:23.359091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.2:57519 (local address 127.20.113.62:33629)
0806 09:28:23.359536 (+ 445us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:23.359542 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:23.360603 (+ 1061us) server_negotiation.cc:407] Connection header received
0806 09:28:23.362361 (+ 1758us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:23.362365 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:23.362434 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:23.362527 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:28:23.363929 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:23.364668 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:23.366085 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:23.366493 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:23.366573 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:23.370396 (+ 3823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:28:23.370401 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:28:23.370404 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:28:23.370433 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:28:23.373252 (+ 2819us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:23.373938 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:23.373945 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:23.373948 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:23.374079 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:23.374508 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:23.374514 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:23.374517 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:23.374774 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:28:23.374931 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:23.375571 (+ 640us) server_negotiation.cc:299] Negotiation successful
0806 09:28:23.375791 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":313,"thread_start_us":207,"threads_started":1}
I20260806 09:28:23.377071 22275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:23.359691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:33629 (local address 127.20.113.2:57519)
0806 09:28:23.360451 (+ 760us) negotiation.cc:107] Waiting for socket to connect
0806 09:28:23.360496 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 09:28:23.362127 (+ 1631us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:28:23.362755 (+ 628us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:28:23.362762 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:28:23.363232 (+ 470us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:28:23.363780 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:23.363797 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:23.364793 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:23.364797 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:23.365965 (+ 1168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:23.365977 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:23.366702 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:23.366710 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:23.367009 (+ 299us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:23.367710 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:28:23.367741 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:28:23.370180 (+ 2439us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:28:23.373407 (+ 3227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:28:23.373415 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:28:23.373429 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:28:23.373818 (+ 389us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:28:23.374190 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:28:23.374195 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:28:23.374198 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:28:23.374398 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:28:23.374912 (+ 514us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:28:23.374919 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:28:23.375223 (+ 304us) client_negotiation.cc:769] Sending connection context
0806 09:28:23.375480 (+ 257us) client_negotiation.cc:240] Negotiation successful
0806 09:28:23.375954 (+ 474us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":292,"threads_started":1}
I20260806 09:28:23.378397 22269 heartbeater.cc:347] Connected to a master server at 127.20.113.62:33629
I20260806 09:28:23.378712 22269 heartbeater.cc:464] Registering TS with master...
I20260806 09:28:23.379402 22269 heartbeater.cc:511] Master 127.20.113.62:33629 requested a full tablet report, sending...
I20260806 09:28:23.380816 21795 ts_manager.cc:194] Registered new tserver with Master: c40b3247fecf4ecba4f70be898f65d7b (127.20.113.2:34717)
I20260806 09:28:23.381608 21795 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.113.2@KRBTEST.COM'} at 127.20.113.2:57519
WARNING: no policy specified for kudu/127.20.113.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.113.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.113.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.3.keytab.
Entry for principal kudu/127.20.113.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.3.keytab.
I20260806 09:28:23.419783 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786008495763054-20932-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786008495763054-20932-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786008495763054-20932-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786008495763054-20932-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.113.3:0
--local_ip_for_outbound_sockets=127.20.113.3
--webserver_interface=127.20.113.3
--webserver_port=0
--tserver_master_addrs=127.20.113.62:33629
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.3.keytab
--principal=kudu/127.20.113.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:41193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:28:23.759017 22279 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:28:23.759294 22279 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:28:23.759469 22279 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:28:23.766846 22279 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:28:23.766957 22279 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:28:23.767134 22279 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.113.3
I20260806 09:28:23.777333 22279 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:41193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786008495763054-20932-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786008495763054-20932-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.3.keytab
--principal=kudu/127.20.113.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.113.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786008495763054-20932-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.113.3
--webserver_port=0
--tserver_master_addrs=127.20.113.62:33629
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.113.3
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786008495763054-20932-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:28:23.778785 22279 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:28:23.780251 22279 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:28:23.794073 22287 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:28:23.793859 22290 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:28:23.793895 22288 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:28:23.795373 22279 server_base.cc:1034] running on GCE node
I20260806 09:28:23.796061 22279 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:28:23.797065 22279 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:28:23.798280 22279 hybrid_clock.cc:648] HybridClock initialized: now 1786008503798237 us; error 30 us; skew 500 ppm
I20260806 09:28:23.798651 22279 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:28:23 dist-test-slave-45dx krb5kdc[21733](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008503, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:28:23.801945 22279 init.cc:377] Logged in from keytab as kudu/127.20.113.3@KRBTEST.COM (short username kudu)
I20260806 09:28:23.803351 22279 webserver.cc:460] Webserver started at http://127.20.113.3:45101/ using document root <none> and password file <none>
I20260806 09:28:23.804050 22279 fs_manager.cc:359] Metadata directory not provided
I20260806 09:28:23.804200 22279 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:28:23.804595 22279 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:28:23.807538 22279 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786008495763054-20932-0/minicluster-data/ts-2/data/instance:
uuid: "30526cea9d654d419aa8f3a0175da9da"
format_stamp: "Formatted at 2026-08-06 09:28:23 on dist-test-slave-45dx"
server_key: "1058b611d1528a5b90c103b513444429"
server_key_iv: "f3fd5b18f45b1f1cf8292b0731d936eb"
server_key_version: "encryptionkey@0"
I20260806 09:28:23.808404 22279 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786008495763054-20932-0/minicluster-data/ts-2/wal/instance:
uuid: "30526cea9d654d419aa8f3a0175da9da"
format_stamp: "Formatted at 2026-08-06 09:28:23 on dist-test-slave-45dx"
server_key: "1058b611d1528a5b90c103b513444429"
server_key_iv: "f3fd5b18f45b1f1cf8292b0731d936eb"
server_key_version: "encryptionkey@0"
I20260806 09:28:23.813565 22279 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 09:28:23.817034 22299 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:28:23.818302 22279 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 09:28:23.818480 22279 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786008495763054-20932-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786008495763054-20932-0/minicluster-data/ts-2/wal
uuid: "30526cea9d654d419aa8f3a0175da9da"
format_stamp: "Formatted at 2026-08-06 09:28:23 on dist-test-slave-45dx"
server_key: "1058b611d1528a5b90c103b513444429"
server_key_iv: "f3fd5b18f45b1f1cf8292b0731d936eb"
server_key_version: "encryptionkey@0"
I20260806 09:28:23.818679 22279 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786008495763054-20932-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786008495763054-20932-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786008495763054-20932-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:28:23.834177 22279 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:28:23.838395 22279 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:28:23.838716 22279 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:28:23.839731 22279 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:28:23.841224 22279 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:28:23.841332 22279 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:28:23.841531 22279 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:28:23.841626 22279 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:28:23.863313 22078 heartbeater.cc:503] Master 127.20.113.62:33629 was elected leader, sending a full tablet report...
I20260806 09:28:23.879958 22279 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.3:45731
I20260806 09:28:23.880031 22456 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.3:45731 every 8 connection(s)
I20260806 09:28:23.881335 22279 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786008495763054-20932-0/minicluster-data/ts-2/data/info.pb
I20260806 09:28:23.890635 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 22279
I20260806 09:28:23.890862 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786008495763054-20932-0/minicluster-data/ts-2/wal/instance
I20260806 09:28:23.891321 20932 external_mini_cluster.cc:1428] Setting key 3a729c3bfb78a071baeb299f396e6e03
Aug 06 09:28:23 dist-test-slave-45dx krb5kdc[21733](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008503, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.3@KRBTEST.COM for kudu/127.20.113.62@KRBTEST.COM
I20260806 09:28:23.902673 22459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:23.883848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.3:58775 (local address 127.20.113.62:33629)
0806 09:28:23.884499 (+ 651us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:23.884503 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:23.885578 (+ 1075us) server_negotiation.cc:407] Connection header received
0806 09:28:23.887645 (+ 2067us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:23.887650 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:23.887766 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:23.887897 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:28:23.889546 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:23.890351 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:23.892101 (+ 1750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:23.892706 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:23.892846 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:23.896545 (+ 3699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:28:23.896551 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:28:23.896556 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:28:23.896599 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:28:23.899335 (+ 2736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:23.900085 (+ 750us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:23.900095 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:23.900097 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:23.900225 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:23.900793 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:23.900802 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:23.900804 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:23.901085 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:28:23.901250 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:23.902049 (+ 799us) server_negotiation.cc:299] Negotiation successful
0806 09:28:23.902286 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":538,"thread_start_us":367,"threads_started":1}
I20260806 09:28:23.903663 22461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:23.884559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:33629 (local address 127.20.113.3:58775)
0806 09:28:23.885315 (+ 756us) negotiation.cc:107] Waiting for socket to connect
0806 09:28:23.885361 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 09:28:23.887367 (+ 2006us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:28:23.888167 (+ 800us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:28:23.888181 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:28:23.888777 (+ 596us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:28:23.889364 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:23.889382 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:23.890530 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:23.890534 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:23.891957 (+ 1423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:23.891970 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:23.892937 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:23.892945 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:23.893183 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:23.893918 (+ 735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:28:23.893941 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:28:23.896282 (+ 2341us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:28:23.899496 (+ 3214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:28:23.899505 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:28:23.899521 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:28:23.899954 (+ 433us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:28:23.900367 (+ 413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:28:23.900372 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:28:23.900376 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:28:23.900661 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:28:23.901223 (+ 562us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:28:23.901231 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:28:23.901591 (+ 360us) client_negotiation.cc:769] Sending connection context
0806 09:28:23.901865 (+ 274us) client_negotiation.cc:240] Negotiation successful
0806 09:28:23.902459 (+ 594us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":521,"thread_start_us":312,"threads_started":1}
I20260806 09:28:23.905241 22457 heartbeater.cc:347] Connected to a master server at 127.20.113.62:33629
I20260806 09:28:23.905627 22457 heartbeater.cc:464] Registering TS with master...
I20260806 09:28:23.906620 22457 heartbeater.cc:511] Master 127.20.113.62:33629 requested a full tablet report, sending...
I20260806 09:28:23.908038 21795 ts_manager.cc:194] Registered new tserver with Master: 30526cea9d654d419aa8f3a0175da9da (127.20.113.3:45731)
I20260806 09:28:23.908885 21795 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.113.3@KRBTEST.COM'} at 127.20.113.3:58775
I20260806 09:28:23.919220 20932 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 09:28:23.938898 22459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:23.926013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55230 (local address 127.20.113.62:33629)
0806 09:28:23.926219 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:23.926228 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:23.926846 (+ 618us) server_negotiation.cc:407] Connection header received
0806 09:28:23.926992 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:23.926996 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:23.927063 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:23.927142 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:28:23.928729 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:23.929437 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:23.930700 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:23.931203 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:23.931298 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:23.933721 (+ 2423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:28:23.933725 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:28:23.933729 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:28:23.933760 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:28:23.936278 (+ 2518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:23.937017 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:23.937021 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:23.937023 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:23.937149 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:23.937557 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:23.937561 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:23.937563 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:23.937865 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:28:23.938073 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:23.938435 (+ 362us) server_negotiation.cc:299] Negotiation successful
0806 09:28:23.938673 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 09:28:23.945277 21795 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55230:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 09:28:23.952867 21795 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 09:28:23.977672 22474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:23.965237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.2:34717 (local address 127.0.0.1:48656)
0806 09:28:23.967466 (+ 2229us) negotiation.cc:107] Waiting for socket to connect
0806 09:28:23.967512 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 09:28:23.967815 (+ 303us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:28:23.969363 (+ 1548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:28:23.969375 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:28:23.969418 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:28:23.969930 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:23.969963 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:23.972701 (+ 2738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:23.972707 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:23.975114 (+ 2407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:23.975130 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:23.976519 (+ 1389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:23.976526 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:23.976806 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:23.976869 (+ 63us) client_negotiation.cc:769] Sending connection context
0806 09:28:23.977028 (+ 159us) client_negotiation.cc:240] Negotiation successful
0806 09:28:23.977257 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1987,"mutex_wait_us":1,"thread_start_us":1838,"threads_started":1}
I20260806 09:28:23.977882 22475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:23.965340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48656 (local address 127.20.113.2:34717)
0806 09:28:23.968525 (+ 3185us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:23.968538 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:23.968576 (+ 38us) server_negotiation.cc:407] Connection header received
0806 09:28:23.968707 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:23.968719 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:23.969025 (+ 306us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:23.970216 (+ 1191us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:28:23.970529 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:23.972527 (+ 1998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:23.975284 (+ 2757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:23.976231 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:23.976471 (+ 240us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:23.976694 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:23.977314 (+ 620us) server_negotiation.cc:299] Negotiation successful
0806 09:28:23.977503 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2977,"thread_start_us":2812,"threads_started":1}
I20260806 09:28:23.980209 22477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:23.965809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.1:38205 (local address 127.0.0.1:59526)
0806 09:28:23.970319 (+ 4510us) negotiation.cc:107] Waiting for socket to connect
0806 09:28:23.970340 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 09:28:23.970500 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:28:23.971928 (+ 1428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:28:23.971933 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:28:23.971957 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:28:23.972562 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:23.972580 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:23.974448 (+ 1868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:23.974455 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:23.977212 (+ 2757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:23.977231 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:23.979268 (+ 2037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:23.979274 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:23.979459 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:23.979483 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 09:28:23.979583 (+ 100us) client_negotiation.cc:240] Negotiation successful
0806 09:28:23.979714 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4342,"mutex_wait_us":73,"thread_start_us":4146,"threads_started":1}
I20260806 09:28:23.980194 22478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:23.966250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59526 (local address 127.20.113.1:38205)
0806 09:28:23.967225 (+ 975us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:23.967238 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:23.971370 (+ 4132us) server_negotiation.cc:407] Connection header received
0806 09:28:23.971541 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:23.971546 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:23.971702 (+ 156us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:23.971906 (+ 204us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:28:23.972783 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:23.974293 (+ 1510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:23.977427 (+ 3134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:23.978974 (+ 1547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:23.979120 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:23.979380 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:23.979666 (+ 286us) server_negotiation.cc:299] Negotiation successful
0806 09:28:23.979838 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":730,"thread_start_us":479,"threads_started":1}
I20260806 09:28:23.980962 22479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:23.968067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.3:45731 (local address 127.0.0.1:60606)
0806 09:28:23.971144 (+ 3077us) negotiation.cc:107] Waiting for socket to connect
0806 09:28:23.971162 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 09:28:23.971262 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:28:23.972263 (+ 1001us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:28:23.972268 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:28:23.972291 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:28:23.972893 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:23.972912 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:23.974946 (+ 2034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:23.974954 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:23.978262 (+ 3308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:23.978275 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:23.980147 (+ 1872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:23.980153 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:23.980360 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:23.980399 (+ 39us) client_negotiation.cc:769] Sending connection context
0806 09:28:23.980478 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 09:28:23.980596 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2934,"mutex_wait_us":949,"thread_start_us":2740,"threads_started":1}
I20260806 09:28:23.981312 22480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:23.968482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60606 (local address 127.20.113.3:45731)
0806 09:28:23.971450 (+ 2968us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:23.971458 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:23.971490 (+ 32us) server_negotiation.cc:407] Connection header received
0806 09:28:23.971582 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:23.971589 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:23.971801 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:23.972000 (+ 199us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:28:23.973146 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:23.974789 (+ 1643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:23.978440 (+ 3651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:23.979614 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:23.979742 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:23.979971 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:23.980805 (+ 834us) server_negotiation.cc:299] Negotiation successful
0806 09:28:23.980975 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2795,"thread_start_us":2634,"threads_started":1}
I20260806 09:28:23.982370 22171 tablet_service.cc:1467] Processing CreateTablet for tablet 9edf1d7f6cc84dd19c9a91dd8fd60b00 (DEFAULT_TABLE table=test-table [id=cd1785fd31f74056b5ccc1b55f09125a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:28:23.984016 22171 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9edf1d7f6cc84dd19c9a91dd8fd60b00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:28:23.983922 21980 tablet_service.cc:1467] Processing CreateTablet for tablet 9edf1d7f6cc84dd19c9a91dd8fd60b00 (DEFAULT_TABLE table=test-table [id=cd1785fd31f74056b5ccc1b55f09125a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:28:23.985205 21980 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9edf1d7f6cc84dd19c9a91dd8fd60b00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:28:23.985301 22367 tablet_service.cc:1467] Processing CreateTablet for tablet 9edf1d7f6cc84dd19c9a91dd8fd60b00 (DEFAULT_TABLE table=test-table [id=cd1785fd31f74056b5ccc1b55f09125a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:28:23.986750 22367 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9edf1d7f6cc84dd19c9a91dd8fd60b00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:28:23.996807 22481 tablet_bootstrap.cc:492] T 9edf1d7f6cc84dd19c9a91dd8fd60b00 P 6561d735f1f1469eabf2d624047dc889: Bootstrap starting.
I20260806 09:28:23.999898 22482 tablet_bootstrap.cc:492] T 9edf1d7f6cc84dd19c9a91dd8fd60b00 P c40b3247fecf4ecba4f70be898f65d7b: Bootstrap starting.
I20260806 09:28:24.000648 22481 tablet_bootstrap.cc:654] T 9edf1d7f6cc84dd19c9a91dd8fd60b00 P 6561d735f1f1469eabf2d624047dc889: Neither blocks nor log segments found. Creating new log.
I20260806 09:28:24.002175 22481 log.cc:824] T 9edf1d7f6cc84dd19c9a91dd8fd60b00 P 6561d735f1f1469eabf2d624047dc889: Log is configured to *not* fsync() on all Append() calls
I20260806 09:28:24.002413 22483 tablet_bootstrap.cc:492] T 9edf1d7f6cc84dd19c9a91dd8fd60b00 P 30526cea9d654d419aa8f3a0175da9da: Bootstrap starting.
I20260806 09:28:24.003844 22482 tablet_bootstrap.cc:654] T 9edf1d7f6cc84dd19c9a91dd8fd60b00 P c40b3247fecf4ecba4f70be898f65d7b: Neither blocks nor log segments found. Creating new log.
I20260806 09:28:24.005100 22482 log.cc:824] T 9edf1d7f6cc84dd19c9a91dd8fd60b00 P c40b3247fecf4ecba4f70be898f65d7b: Log is configured to *not* fsync() on all Append() calls
I20260806 09:28:24.006141 22483 tablet_bootstrap.cc:654] T 9edf1d7f6cc84dd19c9a91dd8fd60b00 P 30526cea9d654d419aa8f3a0175da9da: Neither blocks nor log segments found. Creating new log.
I20260806 09:28:24.006816 22481 tablet_bootstrap.cc:492] T 9edf1d7f6cc84dd19c9a91dd8fd60b00 P 6561d735f1f1469eabf2d624047dc889: No bootstrap required, opened a new log
I20260806 09:28:24.007102 22481 ts_tablet_manager.cc:1397] T 9edf1d7f6cc84dd19c9a91dd8fd60b00 P 6561d735f1f1469eabf2d624047dc889: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.007s
I20260806 09:28:24.007328 22483 log.cc:824] T 9edf1d7f6cc84dd19c9a91dd8fd60b00 P 30526cea9d654d419aa8f3a0175da9da: Log is configured to *not* fsync() on all Append() calls
I20260806 09:28:24.008249 22482 tablet_bootstrap.cc:492] T 9edf1d7f6cc84dd19c9a91dd8fd60b00 P c40b3247fecf4ecba4f70be898f65d7b: No bootstrap required, opened a new log
I20260806 09:28:24.008556 22482 ts_tablet_manager.cc:1397] T 9edf1d7f6cc84dd19c9a91dd8fd60b00 P c40b3247fecf4ecba4f70be898f65d7b: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260806 09:28:24.009963 22483 tablet_bootstrap.cc:492] T 9edf1d7f6cc84dd19c9a91dd8fd60b00 P 30526cea9d654d419aa8f3a0175da9da: No bootstrap required, opened a new log
I20260806 09:28:24.010205 22483 ts_tablet_manager.cc:1397] T 9edf1d7f6cc84dd19c9a91dd8fd60b00 P 30526cea9d654d419aa8f3a0175da9da: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.006s
I20260806 09:28:24.017757 22483 raft_consensus.cc:348] T 9edf1d7f6cc84dd19c9a91dd8fd60b00 P 30526cea9d654d419aa8f3a0175da9da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40b3247fecf4ecba4f70be898f65d7b" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 34717 } } peers { permanent_uuid: "6561d735f1f1469eabf2d624047dc889" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 38205 } } peers { permanent_uuid: "30526cea9d654d419aa8f3a0175da9da" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 45731 } }
I20260806 09:28:24.018137 22483 raft_consensus.cc:374] T 9edf1d7f6cc84dd19c9a91dd8fd60b00 P 30526cea9d654d419aa8f3a0175da9da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:28:24.018252 22483 raft_consensus.cc:729] T 9edf1d7f6cc84dd19c9a91dd8fd60b00 P 30526cea9d654d419aa8f3a0175da9da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30526cea9d654d419aa8f3a0175da9da, State: Initialized, Role: FOLLOWER
I20260806 09:28:24.018428 22481 raft_consensus.cc:348] T 9edf1d7f6cc84dd19c9a91dd8fd60b00 P 6561d735f1f1469eabf2d624047dc889 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40b3247fecf4ecba4f70be898f65d7b" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 34717 } } peers { permanent_uuid: "6561d735f1f1469eabf2d624047dc889" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 38205 } } peers { permanent_uuid: "30526cea9d654d419aa8f3a0175da9da" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 45731 } }
I20260806 09:28:24.018874 22481 raft_consensus.cc:374] T 9edf1d7f6cc84dd19c9a91dd8fd60b00 P 6561d735f1f1469eabf2d624047dc889 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:28:24.018980 22481 raft_consensus.cc:729] T 9edf1d7f6cc84dd19c9a91dd8fd60b00 P 6561d735f1f1469eabf2d624047dc889 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6561d735f1f1469eabf2d624047dc889, State: Initialized, Role: FOLLOWER
I20260806 09:28:24.018905 22483 consensus_queue.cc:260] T 9edf1d7f6cc84dd19c9a91dd8fd60b00 P 30526cea9d654d419aa8f3a0175da9da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40b3247fecf4ecba4f70be898f65d7b" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 34717 } } peers { permanent_uuid: "6561d735f1f1469eabf2d624047dc889" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 38205 } } peers { permanent_uuid: "30526cea9d654d419aa8f3a0175da9da" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 45731 } }
I20260806 09:28:24.019673 22481 consensus_queue.cc:260] T 9edf1d7f6cc84dd19c9a91dd8fd60b00 P 6561d735f1f1469eabf2d624047dc889 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40b3247fecf4ecba4f70be898f65d7b" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 34717 } } peers { permanent_uuid: "6561d735f1f1469eabf2d624047dc889" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 38205 } } peers { permanent_uuid: "30526cea9d654d419aa8f3a0175da9da" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 45731 } }
I20260806 09:28:24.019703 22482 raft_consensus.cc:348] T 9edf1d7f6cc84dd19c9a91dd8fd60b00 P c40b3247fecf4ecba4f70be898f65d7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40b3247fecf4ecba4f70be898f65d7b" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 34717 } } peers { permanent_uuid: "6561d735f1f1469eabf2d624047dc889" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 38205 } } peers { permanent_uuid: "30526cea9d654d419aa8f3a0175da9da" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 45731 } }
I20260806 09:28:24.020154 22457 heartbeater.cc:503] Master 127.20.113.62:33629 was elected leader, sending a full tablet report...
I20260806 09:28:24.020181 22482 raft_consensus.cc:374] T 9edf1d7f6cc84dd19c9a91dd8fd60b00 P c40b3247fecf4ecba4f70be898f65d7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:28:24.020462 22482 raft_consensus.cc:729] T 9edf1d7f6cc84dd19c9a91dd8fd60b00 P c40b3247fecf4ecba4f70be898f65d7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c40b3247fecf4ecba4f70be898f65d7b, State: Initialized, Role: FOLLOWER
I20260806 09:28:24.020860 22483 ts_tablet_manager.cc:1428] T 9edf1d7f6cc84dd19c9a91dd8fd60b00 P 30526cea9d654d419aa8f3a0175da9da: Time spent starting tablet: real 0.011s user 0.006s sys 0.006s
I20260806 09:28:24.021426 22482 consensus_queue.cc:260] T 9edf1d7f6cc84dd19c9a91dd8fd60b00 P c40b3247fecf4ecba4f70be898f65d7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40b3247fecf4ecba4f70be898f65d7b" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 34717 } } peers { permanent_uuid: "6561d735f1f1469eabf2d624047dc889" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 38205 } } peers { permanent_uuid: "30526cea9d654d419aa8f3a0175da9da" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 45731 } }
I20260806 09:28:24.022812 22269 heartbeater.cc:503] Master 127.20.113.62:33629 was elected leader, sending a full tablet report...
I20260806 09:28:24.023764 22482 ts_tablet_manager.cc:1428] T 9edf1d7f6cc84dd19c9a91dd8fd60b00 P c40b3247fecf4ecba4f70be898f65d7b: Time spent starting tablet: real 0.015s user 0.010s sys 0.004s
I20260806 09:28:24.025334 22481 ts_tablet_manager.cc:1428] T 9edf1d7f6cc84dd19c9a91dd8fd60b00 P 6561d735f1f1469eabf2d624047dc889: Time spent starting tablet: real 0.018s user 0.013s sys 0.006s
I20260806 09:28:24.038852 22489 raft_consensus.cc:482] T 9edf1d7f6cc84dd19c9a91dd8fd60b00 P 6561d735f1f1469eabf2d624047dc889 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 09:28:24.039160 22489 raft_consensus.cc:504] T 9edf1d7f6cc84dd19c9a91dd8fd60b00 P 6561d735f1f1469eabf2d624047dc889 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40b3247fecf4ecba4f70be898f65d7b" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 34717 } } peers { permanent_uuid: "6561d735f1f1469eabf2d624047dc889" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 38205 } } peers { permanent_uuid: "30526cea9d654d419aa8f3a0175da9da" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 45731 } }
I20260806 09:28:24.040972 22489 leader_election.cc:288] T 9edf1d7f6cc84dd19c9a91dd8fd60b00 P 6561d735f1f1469eabf2d624047dc889 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c40b3247fecf4ecba4f70be898f65d7b (127.20.113.2:34717), 30526cea9d654d419aa8f3a0175da9da (127.20.113.3:45731)
I20260806 09:28:24.048846 22475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:24.041383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.1:50535 (local address 127.20.113.2:34717)
0806 09:28:24.041625 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:24.041629 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:24.042309 (+ 680us) server_negotiation.cc:407] Connection header received
0806 09:28:24.042570 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:24.042576 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:24.042653 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:24.042718 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:28:24.043664 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:24.044550 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:24.046762 (+ 2212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:24.047907 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:24.048020 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:24.048073 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:24.048519 (+ 446us) server_negotiation.cc:299] Negotiation successful
0806 09:28:24.048615 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260806 09:28:24.049067 22491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:24.041420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.2:34717 (local address 127.20.113.1:50535)
0806 09:28:24.042193 (+ 773us) negotiation.cc:107] Waiting for socket to connect
0806 09:28:24.042215 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 09:28:24.042360 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:28:24.042947 (+ 587us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:28:24.042955 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:28:24.043000 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:28:24.043476 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:24.043489 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:24.044678 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:24.044684 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:24.046603 (+ 1919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:24.046615 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:24.048114 (+ 1499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:24.048120 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:24.048299 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:24.048374 (+ 75us) client_negotiation.cc:769] Sending connection context
0806 09:28:24.048470 (+ 96us) client_negotiation.cc:240] Negotiation successful
0806 09:28:24.048607 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":629,"mutex_wait_us":43,"thread_start_us":452,"threads_started":1}
I20260806 09:28:24.049881 22492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:24.042346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.3:45731 (local address 127.20.113.1:42837)
0806 09:28:24.043012 (+ 666us) negotiation.cc:107] Waiting for socket to connect
0806 09:28:24.043032 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 09:28:24.043248 (+ 216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:28:24.043608 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:28:24.043613 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:28:24.043637 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:28:24.044077 (+ 440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:24.044092 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:24.045498 (+ 1406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:24.045502 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:24.047469 (+ 1967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:24.047484 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:24.049083 (+ 1599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:24.049096 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:24.049312 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:24.049344 (+ 32us) client_negotiation.cc:769] Sending connection context
0806 09:28:24.049449 (+ 105us) client_negotiation.cc:240] Negotiation successful
0806 09:28:24.049579 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":518,"mutex_wait_us":41,"thread_start_us":586,"threads_started":1}
I20260806 09:28:24.049901 22480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:24.042550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.1:42837 (local address 127.20.113.3:45731)
0806 09:28:24.042770 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:24.042777 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:24.043129 (+ 352us) server_negotiation.cc:407] Connection header received
0806 09:28:24.043433 (+ 304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:24.043438 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:24.043496 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:24.043593 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:28:24.044260 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:24.045361 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:24.047620 (+ 2259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:24.048748 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:24.048915 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:24.048985 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:24.049500 (+ 515us) server_negotiation.cc:299] Negotiation successful
0806 09:28:24.049624 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260806 09:28:24.050184 22204 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9edf1d7f6cc84dd19c9a91dd8fd60b00" candidate_uuid: "6561d735f1f1469eabf2d624047dc889" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c40b3247fecf4ecba4f70be898f65d7b" is_pre_election: true
I20260806 09:28:24.050787 22204 raft_consensus.cc:2454] T 9edf1d7f6cc84dd19c9a91dd8fd60b00 P c40b3247fecf4ecba4f70be898f65d7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6561d735f1f1469eabf2d624047dc889 in term 0.
I20260806 09:28:24.050940 22389 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9edf1d7f6cc84dd19c9a91dd8fd60b00" candidate_uuid: "6561d735f1f1469eabf2d624047dc889" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30526cea9d654d419aa8f3a0175da9da" is_pre_election: true
I20260806 09:28:24.051509 22389 raft_consensus.cc:2454] T 9edf1d7f6cc84dd19c9a91dd8fd60b00 P 30526cea9d654d419aa8f3a0175da9da [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6561d735f1f1469eabf2d624047dc889 in term 0.
I20260806 09:28:24.051522 21912 leader_election.cc:302] T 9edf1d7f6cc84dd19c9a91dd8fd60b00 P 6561d735f1f1469eabf2d624047dc889 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6561d735f1f1469eabf2d624047dc889, c40b3247fecf4ecba4f70be898f65d7b; no voters:
I20260806 09:28:24.051949 22489 raft_consensus.cc:2781] T 9edf1d7f6cc84dd19c9a91dd8fd60b00 P 6561d735f1f1469eabf2d624047dc889 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 09:28:24.052135 22489 raft_consensus.cc:482] T 9edf1d7f6cc84dd19c9a91dd8fd60b00 P 6561d735f1f1469eabf2d624047dc889 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 09:28:24.052271 22489 raft_consensus.cc:3037] T 9edf1d7f6cc84dd19c9a91dd8fd60b00 P 6561d735f1f1469eabf2d624047dc889 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:28:24.054632 22489 raft_consensus.cc:504] T 9edf1d7f6cc84dd19c9a91dd8fd60b00 P 6561d735f1f1469eabf2d624047dc889 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40b3247fecf4ecba4f70be898f65d7b" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 34717 } } peers { permanent_uuid: "6561d735f1f1469eabf2d624047dc889" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 38205 } } peers { permanent_uuid: "30526cea9d654d419aa8f3a0175da9da" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 45731 } }
I20260806 09:28:24.055330 22489 leader_election.cc:288] T 9edf1d7f6cc84dd19c9a91dd8fd60b00 P 6561d735f1f1469eabf2d624047dc889 [CANDIDATE]: Term 1 election: Requested vote from peers c40b3247fecf4ecba4f70be898f65d7b (127.20.113.2:34717), 30526cea9d654d419aa8f3a0175da9da (127.20.113.3:45731)
I20260806 09:28:24.055842 22204 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9edf1d7f6cc84dd19c9a91dd8fd60b00" candidate_uuid: "6561d735f1f1469eabf2d624047dc889" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c40b3247fecf4ecba4f70be898f65d7b"
I20260806 09:28:24.055917 22389 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9edf1d7f6cc84dd19c9a91dd8fd60b00" candidate_uuid: "6561d735f1f1469eabf2d624047dc889" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30526cea9d654d419aa8f3a0175da9da"
I20260806 09:28:24.056053 22204 raft_consensus.cc:3037] T 9edf1d7f6cc84dd19c9a91dd8fd60b00 P c40b3247fecf4ecba4f70be898f65d7b [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:28:24.056200 22389 raft_consensus.cc:3037] T 9edf1d7f6cc84dd19c9a91dd8fd60b00 P 30526cea9d654d419aa8f3a0175da9da [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:28:24.058509 22204 raft_consensus.cc:2454] T 9edf1d7f6cc84dd19c9a91dd8fd60b00 P c40b3247fecf4ecba4f70be898f65d7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6561d735f1f1469eabf2d624047dc889 in term 1.
I20260806 09:28:24.058645 22389 raft_consensus.cc:2454] T 9edf1d7f6cc84dd19c9a91dd8fd60b00 P 30526cea9d654d419aa8f3a0175da9da [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6561d735f1f1469eabf2d624047dc889 in term 1.
I20260806 09:28:24.058984 21912 leader_election.cc:302] T 9edf1d7f6cc84dd19c9a91dd8fd60b00 P 6561d735f1f1469eabf2d624047dc889 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6561d735f1f1469eabf2d624047dc889, c40b3247fecf4ecba4f70be898f65d7b; no voters:
I20260806 09:28:24.059332 22489 raft_consensus.cc:2781] T 9edf1d7f6cc84dd19c9a91dd8fd60b00 P 6561d735f1f1469eabf2d624047dc889 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:28:24.059818 22489 raft_consensus.cc:686] T 9edf1d7f6cc84dd19c9a91dd8fd60b00 P 6561d735f1f1469eabf2d624047dc889 [term 1 LEADER]: Becoming Leader. State: Replica: 6561d735f1f1469eabf2d624047dc889, State: Running, Role: LEADER
I20260806 09:28:24.060384 22489 consensus_queue.cc:237] T 9edf1d7f6cc84dd19c9a91dd8fd60b00 P 6561d735f1f1469eabf2d624047dc889 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40b3247fecf4ecba4f70be898f65d7b" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 34717 } } peers { permanent_uuid: "6561d735f1f1469eabf2d624047dc889" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 38205 } } peers { permanent_uuid: "30526cea9d654d419aa8f3a0175da9da" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 45731 } }
I20260806 09:28:24.065238 21795 catalog_manager.cc:5495] T 9edf1d7f6cc84dd19c9a91dd8fd60b00 P 6561d735f1f1469eabf2d624047dc889 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6561d735f1f1469eabf2d624047dc889 (127.20.113.1). New cstate: current_term: 1 leader_uuid: "6561d735f1f1469eabf2d624047dc889" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40b3247fecf4ecba4f70be898f65d7b" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 34717 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6561d735f1f1469eabf2d624047dc889" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 38205 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "30526cea9d654d419aa8f3a0175da9da" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 45731 } health_report { overall_health: UNKNOWN } } }
W20260806 09:28:24.082502 22079 tablet.cc:2369] T 9edf1d7f6cc84dd19c9a91dd8fd60b00 P 6561d735f1f1469eabf2d624047dc889: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 06 09:28:24 dist-test-slave-45dx krb5kdc[21733](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008501, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.113.1@KRBTEST.COM
I20260806 09:28:24.105795 22478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:24.090692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59540 (local address 127.20.113.1:38205)
0806 09:28:24.090937 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:24.090946 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:24.091458 (+ 512us) server_negotiation.cc:407] Connection header received
0806 09:28:24.091603 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:24.091607 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:24.091675 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:24.091769 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:28:24.093484 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:24.094249 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:24.095434 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:24.095857 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:24.095954 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:24.099992 (+ 4038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:28:24.100006 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:28:24.100022 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:28:24.100061 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:28:24.103002 (+ 2941us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:24.103667 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:24.103675 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:24.103677 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:24.103930 (+ 253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:24.104275 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:24.104292 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:24.104294 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:24.104722 (+ 428us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:28:24.104945 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:24.105213 (+ 268us) server_negotiation.cc:299] Negotiation successful
0806 09:28:24.105427 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
W20260806 09:28:24.108623 22270 tablet.cc:2369] T 9edf1d7f6cc84dd19c9a91dd8fd60b00 P c40b3247fecf4ecba4f70be898f65d7b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 09:28:24.133183 22458 tablet.cc:2369] T 9edf1d7f6cc84dd19c9a91dd8fd60b00 P 30526cea9d654d419aa8f3a0175da9da: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 06 09:28:24 dist-test-slave-45dx krb5kdc[21733](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008504, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 09:28:24 dist-test-slave-45dx krb5kdc[21733](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008504, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.20.113.1@KRBTEST.COM
I20260806 09:28:24.164047 22478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:24.149569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59544 (local address 127.20.113.1:38205)
0806 09:28:24.149808 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:24.149816 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:24.150099 (+ 283us) server_negotiation.cc:407] Connection header received
0806 09:28:24.150327 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:24.150331 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:24.150397 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:24.150504 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:28:24.152111 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:24.153019 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:24.154367 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:24.154851 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:24.155001 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:24.158894 (+ 3893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:28:24.158899 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:28:24.158902 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:28:24.158934 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:28:24.161440 (+ 2506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:24.162069 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:24.162076 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:24.162079 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:24.162220 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:24.162634 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:24.162640 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:24.162643 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:24.162916 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:28:24.163214 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:24.163469 (+ 255us) server_negotiation.cc:299] Negotiation successful
0806 09:28:24.163686 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
Aug 06 09:28:24 dist-test-slave-45dx krb5kdc[21733](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008504, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 09:28:24 dist-test-slave-45dx krb5kdc[21733](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008504, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.113.1@KRBTEST.COM
I20260806 09:28:24.221807 22478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:24.207158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59546 (local address 127.20.113.1:38205)
0806 09:28:24.207362 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:24.207370 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:24.207702 (+ 332us) server_negotiation.cc:407] Connection header received
0806 09:28:24.207915 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:24.207919 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:24.208003 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:24.208110 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:28:24.209548 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:24.210312 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:24.211541 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:24.211978 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:24.212129 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:24.216267 (+ 4138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:28:24.216272 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:28:24.216275 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:28:24.216358 (+ 83us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:28:24.218974 (+ 2616us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:24.219619 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:24.219625 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:24.219628 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:24.219766 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:24.220144 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:24.220150 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:24.220153 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:24.220501 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:28:24.220788 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:24.221105 (+ 317us) server_negotiation.cc:299] Negotiation successful
0806 09:28:24.221353 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
W20260806 09:28:24.224079 21944 tablet_service.cc:2908] Rejecting scan request for tablet 9edf1d7f6cc84dd19c9a91dd8fd60b00: Uninitialized: clean time has not yet been initialized
I20260806 09:28:24.227386 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 21886
I20260806 09:28:24.239795 22478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:24.228389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59558 (local address 127.20.113.1:38205)
0806 09:28:24.228576 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:24.228584 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:24.228948 (+ 364us) server_negotiation.cc:407] Connection header received
0806 09:28:24.229052 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:24.229055 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:24.229111 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:24.229186 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:28:24.230319 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:24.231013 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:24.232124 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:24.232593 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:24.232685 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:24.234838 (+ 2153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:28:24.234842 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:28:24.234845 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:28:24.234870 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:28:24.237537 (+ 2667us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:24.238157 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:24.238161 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:24.238163 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:24.238278 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:24.238575 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:24.238578 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:24.238579 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:24.238815 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:28:24.238998 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:24.239226 (+ 228us) server_negotiation.cc:299] Negotiation successful
0806 09:28:24.239421 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 09:28:24.240521 22072 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:28:24.339255 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 21886
I20260806 09:28:24.369915 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 22086
Aug 06 09:28:24 dist-test-slave-45dx krb5kdc[21733](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008504, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.113.2@KRBTEST.COM
I20260806 09:28:24.385493 22475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:24.370648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48670 (local address 127.20.113.2:34717)
0806 09:28:24.370815 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:24.370819 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:24.370916 (+ 97us) server_negotiation.cc:407] Connection header received
0806 09:28:24.371112 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:24.371116 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:24.371206 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:24.371287 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:28:24.372760 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:24.373532 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:24.375324 (+ 1792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:24.375701 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:24.375784 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:24.380141 (+ 4357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:28:24.380153 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:28:24.380166 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:28:24.380197 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:28:24.382821 (+ 2624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:24.383528 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:24.383536 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:24.383539 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:24.383681 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:24.384135 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:24.384139 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:24.384140 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:24.384605 (+ 465us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:28:24.384854 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:24.385100 (+ 246us) server_negotiation.cc:299] Negotiation successful
0806 09:28:24.385259 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 09:28:24.386142 22260 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:28:24.474460 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 22086
I20260806 09:28:24.500515 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 22279
Aug 06 09:28:24 dist-test-slave-45dx krb5kdc[21733](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008504, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.113.3@KRBTEST.COM
I20260806 09:28:24.515128 22480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:24.501328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60620 (local address 127.20.113.3:45731)
0806 09:28:24.501523 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:24.501528 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:24.501550 (+ 22us) server_negotiation.cc:407] Connection header received
0806 09:28:24.501678 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:24.501682 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:24.501765 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:24.501879 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:28:24.503113 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:24.503885 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:24.505208 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:24.505614 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:24.505720 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:24.509775 (+ 4055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:28:24.509793 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:28:24.509810 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:28:24.509844 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:28:24.512732 (+ 2888us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:24.513344 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:24.513352 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:24.513354 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:24.513482 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:24.513800 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:24.513804 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:24.513806 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:24.514149 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:28:24.514389 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:24.514586 (+ 197us) server_negotiation.cc:299] Negotiation successful
0806 09:28:24.514769 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260806 09:28:24.515799 22449 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:28:24.604383 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 22279
I20260806 09:28:24.634620 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 21749
I20260806 09:28:24.635778 21837 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:28:24.746124 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 21749
2026-08-06T09:28:24Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (4480 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786008495763054-20932-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 09:28:24 dist-test-slave-45dx krb5kdc[22520](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 09:28:24 dist-test-slave-45dx krb5kdc[22520](info): set up 2 sockets
Aug 06 09:28:24 dist-test-slave-45dx krb5kdc[22520](info): commencing operation
krb5kdc: starting...
W20260806 09:28:26.021035 20932 mini_kdc.cc:121] Time spent starting KDC: real 1.223s user 0.004s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786008495763054-20932-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786008495763054-20932-0/krb5kdc/test-user.keytab.
Aug 06 09:28:26 dist-test-slave-45dx krb5kdc[22520](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008506, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T09:28:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T09:28:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.113.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.113.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.113.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab.
Entry for principal kudu/127.20.113.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab.
I20260806 09:28:26.239557 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786008495763054-20932-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786008495763054-20932-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786008495763054-20932-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.113.62:37651
--webserver_interface=127.20.113.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab
--principal=kudu/127.20.113.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:43381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.113.62:37651
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:28:26.579051 22540 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:28:26.579382 22540 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:28:26.579545 22540 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:28:26.586879 22540 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 09:28:26.587020 22540 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 09:28:26.587103 22540 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:28:26.587216 22540 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:28:26.587299 22540 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 09:28:26.587388 22540 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 09:28:26.597431 22540 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:43381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786008495763054-20932-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786008495763054-20932-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.113.62:37651
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab
--principal=kudu/127.20.113.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.113.62:37651
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.113.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786008495763054-20932-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:28:26.598728 22540 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:28:26.600097 22540 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:28:26.612030 22547 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:28:26.612068 22548 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:28:26.612108 22550 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:28:26.613312 22540 server_base.cc:1034] running on GCE node
I20260806 09:28:26.614089 22540 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:28:26.615507 22540 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:28:26.616755 22540 hybrid_clock.cc:648] HybridClock initialized: now 1786008506616700 us; error 52 us; skew 500 ppm
I20260806 09:28:26.617092 22540 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:28:26 dist-test-slave-45dx krb5kdc[22520](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008506, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:28:26.620420 22540 init.cc:377] Logged in from keytab as kudu/127.20.113.62@KRBTEST.COM (short username kudu)
I20260806 09:28:26.621910 22540 webserver.cc:460] Webserver started at http://127.20.113.62:37183/ using document root <none> and password file <none>
I20260806 09:28:26.622632 22540 fs_manager.cc:359] Metadata directory not provided
I20260806 09:28:26.622777 22540 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:28:26.623133 22540 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:28:26.626288 22540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786008495763054-20932-0/minicluster-data/master-0/data/instance:
uuid: "aefb46d3774747bca7b9baf9042e72a3"
format_stamp: "Formatted at 2026-08-06 09:28:26 on dist-test-slave-45dx"
server_key: "9b342e8f297e0dd4d67f0730bd17644f"
server_key_iv: "20d2b012628876c9d88c1f8ac66d9326"
server_key_version: "encryptionkey@0"
I20260806 09:28:26.627094 22540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786008495763054-20932-0/minicluster-data/master-0/wal/instance:
uuid: "aefb46d3774747bca7b9baf9042e72a3"
format_stamp: "Formatted at 2026-08-06 09:28:26 on dist-test-slave-45dx"
server_key: "9b342e8f297e0dd4d67f0730bd17644f"
server_key_iv: "20d2b012628876c9d88c1f8ac66d9326"
server_key_version: "encryptionkey@0"
I20260806 09:28:26.632407 22540 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260806 09:28:26.635880 22559 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:28:26.637244 22540 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260806 09:28:26.637432 22540 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786008495763054-20932-0/minicluster-data/master-0/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786008495763054-20932-0/minicluster-data/master-0/wal
uuid: "aefb46d3774747bca7b9baf9042e72a3"
format_stamp: "Formatted at 2026-08-06 09:28:26 on dist-test-slave-45dx"
server_key: "9b342e8f297e0dd4d67f0730bd17644f"
server_key_iv: "20d2b012628876c9d88c1f8ac66d9326"
server_key_version: "encryptionkey@0"
I20260806 09:28:26.637670 22540 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786008495763054-20932-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786008495763054-20932-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786008495763054-20932-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:28:26.660389 22540 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:28:26.664435 22540 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:28:26.664707 22540 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:28:26.687326 22540 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.62:37651
I20260806 09:28:26.687351 22645 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.62:37651 every 8 connection(s)
I20260806 09:28:26.688793 22540 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
I20260806 09:28:26.690685 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 22540
I20260806 09:28:26.690934 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786008495763054-20932-0/minicluster-data/master-0/wal/instance
I20260806 09:28:26.691634 20932 external_mini_cluster.cc:1428] Setting key b11e04a5035427fefc552d1a973d4e65
I20260806 09:28:26.692942 22647 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 06 09:28:26 dist-test-slave-45dx krb5kdc[22520](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008506, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.113.62@KRBTEST.COM
I20260806 09:28:26.708194 22647 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aefb46d3774747bca7b9baf9042e72a3: Bootstrap starting.
I20260806 09:28:26.711817 22647 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aefb46d3774747bca7b9baf9042e72a3: Neither blocks nor log segments found. Creating new log.
I20260806 09:28:26.713181 22647 log.cc:824] T 00000000000000000000000000000000 P aefb46d3774747bca7b9baf9042e72a3: Log is configured to *not* fsync() on all Append() calls
I20260806 09:28:26.715687 22651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:26.693708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52934 (local address 127.20.113.62:37651)
0806 09:28:26.694584 (+ 876us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:26.694596 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:26.694638 (+ 42us) server_negotiation.cc:407] Connection header received
0806 09:28:26.696347 (+ 1709us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:26.696389 (+ 42us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:26.696798 (+ 409us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:26.697159 (+ 361us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:28:26.699062 (+ 1903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:26.700545 (+ 1483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:26.702248 (+ 1703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:26.702900 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:26.703034 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:26.708129 (+ 5095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:28:26.708137 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:28:26.708153 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:28:26.708200 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:28:26.711345 (+ 3145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:26.712167 (+ 822us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:26.712174 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:26.712180 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:26.712451 (+ 271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:26.712851 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:26.712854 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:26.712856 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:26.713441 (+ 585us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:28:26.713658 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:26.714241 (+ 583us) server_negotiation.cc:299] Negotiation successful
0806 09:28:26.714700 (+ 459us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":648,"thread_start_us":392,"threads_started":1}
I20260806 09:28:26.716282 22647 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aefb46d3774747bca7b9baf9042e72a3: No bootstrap required, opened a new log
I20260806 09:28:26.725055 22647 raft_consensus.cc:348] T 00000000000000000000000000000000 P aefb46d3774747bca7b9baf9042e72a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aefb46d3774747bca7b9baf9042e72a3" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 37651 } }
I20260806 09:28:26.725575 22647 raft_consensus.cc:374] T 00000000000000000000000000000000 P aefb46d3774747bca7b9baf9042e72a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:28:26.725680 22647 raft_consensus.cc:729] T 00000000000000000000000000000000 P aefb46d3774747bca7b9baf9042e72a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aefb46d3774747bca7b9baf9042e72a3, State: Initialized, Role: FOLLOWER
I20260806 09:28:26.726194 22647 consensus_queue.cc:260] T 00000000000000000000000000000000 P aefb46d3774747bca7b9baf9042e72a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aefb46d3774747bca7b9baf9042e72a3" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 37651 } }
I20260806 09:28:26.726397 22647 raft_consensus.cc:388] T 00000000000000000000000000000000 P aefb46d3774747bca7b9baf9042e72a3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 09:28:26.726497 22647 raft_consensus.cc:482] T 00000000000000000000000000000000 P aefb46d3774747bca7b9baf9042e72a3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 09:28:26.726614 22647 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aefb46d3774747bca7b9baf9042e72a3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:28:26.728883 22647 raft_consensus.cc:504] T 00000000000000000000000000000000 P aefb46d3774747bca7b9baf9042e72a3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aefb46d3774747bca7b9baf9042e72a3" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 37651 } }
I20260806 09:28:26.729321 22647 leader_election.cc:302] T 00000000000000000000000000000000 P aefb46d3774747bca7b9baf9042e72a3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aefb46d3774747bca7b9baf9042e72a3; no voters:
I20260806 09:28:26.729952 22647 leader_election.cc:288] T 00000000000000000000000000000000 P aefb46d3774747bca7b9baf9042e72a3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 09:28:26.730074 22654 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aefb46d3774747bca7b9baf9042e72a3 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:28:26.731247 22654 raft_consensus.cc:686] T 00000000000000000000000000000000 P aefb46d3774747bca7b9baf9042e72a3 [term 1 LEADER]: Becoming Leader. State: Replica: aefb46d3774747bca7b9baf9042e72a3, State: Running, Role: LEADER
I20260806 09:28:26.731767 22654 consensus_queue.cc:237] T 00000000000000000000000000000000 P aefb46d3774747bca7b9baf9042e72a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aefb46d3774747bca7b9baf9042e72a3" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 37651 } }
I20260806 09:28:26.732352 22647 sys_catalog.cc:564] T 00000000000000000000000000000000 P aefb46d3774747bca7b9baf9042e72a3 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 09:28:26.734771 22655 sys_catalog.cc:455] T 00000000000000000000000000000000 P aefb46d3774747bca7b9baf9042e72a3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "aefb46d3774747bca7b9baf9042e72a3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aefb46d3774747bca7b9baf9042e72a3" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 37651 } } }
I20260806 09:28:26.736059 22655 sys_catalog.cc:458] T 00000000000000000000000000000000 P aefb46d3774747bca7b9baf9042e72a3 [sys.catalog]: This master's current role is: LEADER
I20260806 09:28:26.735797 22657 sys_catalog.cc:455] T 00000000000000000000000000000000 P aefb46d3774747bca7b9baf9042e72a3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aefb46d3774747bca7b9baf9042e72a3. Latest consensus state: current_term: 1 leader_uuid: "aefb46d3774747bca7b9baf9042e72a3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aefb46d3774747bca7b9baf9042e72a3" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 37651 } } }
I20260806 09:28:26.740712 22657 sys_catalog.cc:458] T 00000000000000000000000000000000 P aefb46d3774747bca7b9baf9042e72a3 [sys.catalog]: This master's current role is: LEADER
I20260806 09:28:26.741360 22666 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 09:28:26.746330 22666 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 09:28:26.754469 22666 catalog_manager.cc:1347] Generated new cluster ID: ba76807318364a9ab073247a0cb651fa
I20260806 09:28:26.754585 22666 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 09:28:26.763381 22666 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 09:28:26.765054 22666 catalog_manager.cc:1504] Loading token signing keys...
I20260806 09:28:26.770802 22666 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aefb46d3774747bca7b9baf9042e72a3: Generated new TSK 0
I20260806 09:28:26.771584 22666 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.113.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.113.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.113.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.1.keytab.
Entry for principal kudu/127.20.113.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.1.keytab.
I20260806 09:28:26.825887 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786008495763054-20932-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786008495763054-20932-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786008495763054-20932-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786008495763054-20932-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.113.1:0
--local_ip_for_outbound_sockets=127.20.113.1
--webserver_interface=127.20.113.1
--webserver_port=0
--tserver_master_addrs=127.20.113.62:37651
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.1.keytab
--principal=kudu/127.20.113.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:43381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:28:27.163707 22684 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:28:27.163982 22684 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:28:27.164155 22684 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:28:27.171223 22684 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:28:27.171320 22684 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:28:27.171401 22684 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 09:28:27.171568 22684 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.113.1
I20260806 09:28:27.180953 22684 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:43381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786008495763054-20932-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786008495763054-20932-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.1.keytab
--principal=kudu/127.20.113.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.113.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786008495763054-20932-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.113.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.20.113.62:37651
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.113.1
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786008495763054-20932-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:28:27.182255 22684 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:28:27.183594 22684 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:28:27.197286 22689 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:28:27.197222 22690 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:28:27.197317 22693 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:28:27.198401 22684 server_base.cc:1034] running on GCE node
I20260806 09:28:27.199100 22684 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:28:27.200003 22684 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:28:27.201243 22684 hybrid_clock.cc:648] HybridClock initialized: now 1786008507201177 us; error 41 us; skew 500 ppm
I20260806 09:28:27.201550 22684 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:28:27 dist-test-slave-45dx krb5kdc[22520](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008507, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:28:27.209669 22684 init.cc:377] Logged in from keytab as kudu/127.20.113.1@KRBTEST.COM (short username kudu)
I20260806 09:28:27.211156 22684 webserver.cc:460] Webserver started at http://127.20.113.1:42581/ using document root <none> and password file <none>
I20260806 09:28:27.211833 22684 fs_manager.cc:359] Metadata directory not provided
I20260806 09:28:27.211922 22684 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:28:27.212203 22684 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:28:27.215309 22684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786008495763054-20932-0/minicluster-data/ts-0/data/instance:
uuid: "63e98f6f427648d298613653395a2702"
format_stamp: "Formatted at 2026-08-06 09:28:27 on dist-test-slave-45dx"
server_key: "ee7dd342c3f880d500b5aabf549f9a5f"
server_key_iv: "64faedd6df9bcb268016fd8c6b36a267"
server_key_version: "encryptionkey@0"
I20260806 09:28:27.216192 22684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786008495763054-20932-0/minicluster-data/ts-0/wal/instance:
uuid: "63e98f6f427648d298613653395a2702"
format_stamp: "Formatted at 2026-08-06 09:28:27 on dist-test-slave-45dx"
server_key: "ee7dd342c3f880d500b5aabf549f9a5f"
server_key_iv: "64faedd6df9bcb268016fd8c6b36a267"
server_key_version: "encryptionkey@0"
I20260806 09:28:27.221517 22684 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260806 09:28:27.225085 22704 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:28:27.226524 22684 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 09:28:27.226655 22684 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786008495763054-20932-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786008495763054-20932-0/minicluster-data/ts-0/wal
uuid: "63e98f6f427648d298613653395a2702"
format_stamp: "Formatted at 2026-08-06 09:28:27 on dist-test-slave-45dx"
server_key: "ee7dd342c3f880d500b5aabf549f9a5f"
server_key_iv: "64faedd6df9bcb268016fd8c6b36a267"
server_key_version: "encryptionkey@0"
I20260806 09:28:27.226833 22684 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786008495763054-20932-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786008495763054-20932-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786008495763054-20932-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:28:27.260424 22684 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:28:27.264598 22684 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:28:27.264858 22684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:28:27.267827 22684 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:28:27.267948 22684 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:28:27.268119 22684 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:28:27.268208 22684 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 09:28:27 dist-test-slave-45dx krb5kdc[22520](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008507, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.1@KRBTEST.COM for kudu/127.20.113.62@KRBTEST.COM
I20260806 09:28:27.293269 22716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:27.268492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.1:51759 (local address 127.20.113.62:37651)
0806 09:28:27.269232 (+ 740us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:27.269236 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:27.270327 (+ 1091us) server_negotiation.cc:407] Connection header received
0806 09:28:27.272673 (+ 2346us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:27.272679 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:27.272772 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:27.272925 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:28:27.274805 (+ 1880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:27.275851 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:27.278622 (+ 2771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:27.279259 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:27.279406 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:27.283921 (+ 4515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:28:27.283930 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:28:27.283935 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:28:27.283984 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:28:27.287687 (+ 3703us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:27.288856 (+ 1169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:27.288864 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:27.288869 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:27.289056 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:27.289708 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:27.289718 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:27.289724 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:27.290166 (+ 442us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:28:27.290345 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:27.292569 (+ 2224us) server_negotiation.cc:299] Negotiation successful
0806 09:28:27.292819 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":627,"thread_start_us":519,"threads_started":1}
I20260806 09:28:27.294061 22717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:27.269177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:37651 (local address 127.20.113.1:51759)
0806 09:28:27.270160 (+ 983us) negotiation.cc:107] Waiting for socket to connect
0806 09:28:27.270207 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 09:28:27.272407 (+ 2200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:28:27.273151 (+ 744us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:28:27.273158 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:28:27.273688 (+ 530us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:28:27.274550 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:27.274581 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:27.276043 (+ 1462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:27.276048 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:27.278401 (+ 2353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:27.278421 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:27.279526 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:27.279534 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:27.279741 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:27.280496 (+ 755us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:28:27.280548 (+ 52us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:28:27.283633 (+ 3085us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:28:27.287868 (+ 4235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:28:27.287882 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:28:27.287900 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:28:27.288706 (+ 806us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:28:27.289177 (+ 471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:28:27.289182 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:28:27.289186 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:28:27.289526 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:28:27.290437 (+ 911us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:28:27.290449 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:28:27.291037 (+ 588us) client_negotiation.cc:769] Sending connection context
0806 09:28:27.291401 (+ 364us) client_negotiation.cc:240] Negotiation successful
0806 09:28:27.292086 (+ 685us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":676,"thread_start_us":403,"threads_started":1}
I20260806 09:28:27.310479 22684 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.1:34967
I20260806 09:28:27.310485 22874 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.1:34967 every 8 connection(s)
I20260806 09:28:27.312052 22684 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786008495763054-20932-0/minicluster-data/ts-0/data/info.pb
I20260806 09:28:27.314421 22875 heartbeater.cc:347] Connected to a master server at 127.20.113.62:37651
I20260806 09:28:27.314651 22875 heartbeater.cc:464] Registering TS with master...
I20260806 09:28:27.315255 22875 heartbeater.cc:511] Master 127.20.113.62:37651 requested a full tablet report, sending...
I20260806 09:28:27.317271 22583 ts_manager.cc:194] Registered new tserver with Master: 63e98f6f427648d298613653395a2702 (127.20.113.1:34967)
I20260806 09:28:27.317715 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 22684
I20260806 09:28:27.317940 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786008495763054-20932-0/minicluster-data/ts-0/wal/instance
I20260806 09:28:27.318768 20932 external_mini_cluster.cc:1428] Setting key c457f968e9d2aaff2a9f80957eb5b075
I20260806 09:28:27.319406 22583 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.113.1@KRBTEST.COM'} at 127.20.113.1:51759
I20260806 09:28:27.319720 22716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:27.301944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.1:33195 (local address 127.20.113.62:37651)
0806 09:28:27.302149 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:27.302155 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:27.305112 (+ 2957us) server_negotiation.cc:407] Connection header received
0806 09:28:27.305335 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:27.305342 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:27.305451 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:27.305575 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:28:27.307285 (+ 1710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:27.308429 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:27.310733 (+ 2304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:27.311402 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:27.311545 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:27.313907 (+ 2362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:28:27.313914 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:28:27.313918 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:28:27.313956 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:28:27.316842 (+ 2886us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:27.317509 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:27.317515 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:27.317518 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:27.317662 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:27.318115 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:27.318118 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:27.318120 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:27.318488 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:28:27.318642 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:27.319056 (+ 414us) server_negotiation.cc:299] Negotiation successful
0806 09:28:27.319275 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260806 09:28:27.319736 22841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:27.302090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:37651 (local address 127.20.113.1:33195)
0806 09:28:27.304972 (+ 2882us) negotiation.cc:107] Waiting for socket to connect
0806 09:28:27.305003 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0806 09:28:27.305183 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:28:27.305627 (+ 444us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:28:27.305635 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:28:27.306425 (+ 790us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:28:27.307092 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:27.307117 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:27.308571 (+ 1454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:27.308576 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:27.310553 (+ 1977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:27.310574 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:27.311608 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:27.311633 (+ 25us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:27.311769 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:27.312456 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:28:27.312492 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:28:27.313713 (+ 1221us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:28:27.316975 (+ 3262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:28:27.316981 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:28:27.316985 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:28:27.317353 (+ 368us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:28:27.317772 (+ 419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:28:27.317777 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:28:27.317781 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:28:27.317940 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:28:27.318603 (+ 663us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:28:27.318610 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:28:27.318784 (+ 174us) client_negotiation.cc:769] Sending connection context
0806 09:28:27.318944 (+ 160us) client_negotiation.cc:240] Negotiation successful
0806 09:28:27.319269 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2665,"thread_start_us":2433,"threads_started":1}
WARNING: no policy specified for kudu/127.20.113.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.113.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.113.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.2.keytab.
Entry for principal kudu/127.20.113.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.2.keytab.
I20260806 09:28:27.371491 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786008495763054-20932-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786008495763054-20932-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786008495763054-20932-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786008495763054-20932-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.113.2:0
--local_ip_for_outbound_sockets=127.20.113.2
--webserver_interface=127.20.113.2
--webserver_port=0
--tserver_master_addrs=127.20.113.62:37651
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.2.keytab
--principal=kudu/127.20.113.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:43381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:28:27.708882 22880 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:28:27.709168 22880 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:28:27.709349 22880 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:28:27.716794 22880 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:28:27.716943 22880 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:28:27.717099 22880 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 09:28:27.717226 22880 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.113.2
I20260806 09:28:27.727291 22880 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:43381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786008495763054-20932-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786008495763054-20932-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.2.keytab
--principal=kudu/127.20.113.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.113.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786008495763054-20932-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.113.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.20.113.62:37651
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.113.2
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786008495763054-20932-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:28:27.728674 22880 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:28:27.730121 22880 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:28:27.743543 22894 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:28:27.743583 22891 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:28:27.743831 22886 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:28:27.744709 22880 server_base.cc:1034] running on GCE node
I20260806 09:28:27.745270 22880 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:28:27.746122 22880 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:28:27.747339 22880 hybrid_clock.cc:648] HybridClock initialized: now 1786008507747278 us; error 32 us; skew 500 ppm
I20260806 09:28:27.747645 22880 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:28:27 dist-test-slave-45dx krb5kdc[22520](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008507, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:28:27.750732 22880 init.cc:377] Logged in from keytab as kudu/127.20.113.2@KRBTEST.COM (short username kudu)
I20260806 09:28:27.752236 22880 webserver.cc:460] Webserver started at http://127.20.113.2:36571/ using document root <none> and password file <none>
I20260806 09:28:27.752954 22880 fs_manager.cc:359] Metadata directory not provided
I20260806 09:28:27.753086 22880 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:28:27.753407 22880 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:28:27.756222 22880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786008495763054-20932-0/minicluster-data/ts-1/data/instance:
uuid: "7b0735292a6c448891ef04ecf4df52a2"
format_stamp: "Formatted at 2026-08-06 09:28:27 on dist-test-slave-45dx"
server_key: "485163991673b97909a4b2399a524f44"
server_key_iv: "e4d1eac00f402a45bd0e34780009ade9"
server_key_version: "encryptionkey@0"
I20260806 09:28:27.757122 22880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786008495763054-20932-0/minicluster-data/ts-1/wal/instance:
uuid: "7b0735292a6c448891ef04ecf4df52a2"
format_stamp: "Formatted at 2026-08-06 09:28:27 on dist-test-slave-45dx"
server_key: "485163991673b97909a4b2399a524f44"
server_key_iv: "e4d1eac00f402a45bd0e34780009ade9"
server_key_version: "encryptionkey@0"
I20260806 09:28:27.762351 22880 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 09:28:27.765738 22903 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:28:27.767007 22880 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 09:28:27.767184 22880 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786008495763054-20932-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786008495763054-20932-0/minicluster-data/ts-1/wal
uuid: "7b0735292a6c448891ef04ecf4df52a2"
format_stamp: "Formatted at 2026-08-06 09:28:27 on dist-test-slave-45dx"
server_key: "485163991673b97909a4b2399a524f44"
server_key_iv: "e4d1eac00f402a45bd0e34780009ade9"
server_key_version: "encryptionkey@0"
I20260806 09:28:27.767398 22880 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786008495763054-20932-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786008495763054-20932-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786008495763054-20932-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:28:27.782073 22880 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:28:27.786223 22880 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:28:27.786510 22880 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:28:27.789332 22880 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:28:27.789465 22880 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:28:27.789639 22880 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:28:27.789716 22880 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 09:28:27 dist-test-slave-45dx krb5kdc[22520](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008507, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.2@KRBTEST.COM for kudu/127.20.113.62@KRBTEST.COM
I20260806 09:28:27.812233 22716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:27.789677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.2:35253 (local address 127.20.113.62:37651)
0806 09:28:27.789917 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:27.789925 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:27.791383 (+ 1458us) server_negotiation.cc:407] Connection header received
0806 09:28:27.793649 (+ 2266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:27.793656 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:27.793759 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:27.793912 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:28:27.795767 (+ 1855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:27.796955 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:27.799689 (+ 2734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:27.800154 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:27.800297 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:27.805195 (+ 4898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:28:27.805200 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:28:27.805203 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:28:27.805237 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:28:27.808780 (+ 3543us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:27.809844 (+ 1064us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:27.809849 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:27.809851 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:27.809970 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:27.810409 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:27.810415 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:27.810418 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:27.810717 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:28:27.810886 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:27.811624 (+ 738us) server_negotiation.cc:299] Negotiation successful
0806 09:28:27.811859 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260806 09:28:27.813972 22915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:27.790265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:37651 (local address 127.20.113.2:35253)
0806 09:28:27.791177 (+ 912us) negotiation.cc:107] Waiting for socket to connect
0806 09:28:27.791234 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0806 09:28:27.793352 (+ 2118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:28:27.794207 (+ 855us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:28:27.794217 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:28:27.794975 (+ 758us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:28:27.795587 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:27.795606 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:27.797113 (+ 1507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:27.797120 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:27.799531 (+ 2411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:27.799547 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:27.800442 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:27.800451 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:27.800718 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:27.801569 (+ 851us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:28:27.801591 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:28:27.804904 (+ 3313us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:28:27.808940 (+ 4036us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:28:27.808951 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:28:27.808970 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:28:27.809675 (+ 705us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:28:27.810086 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:28:27.810091 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:28:27.810099 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:28:27.810290 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:28:27.810866 (+ 576us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:28:27.810875 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:28:27.811223 (+ 348us) client_negotiation.cc:769] Sending connection context
0806 09:28:27.811525 (+ 302us) client_negotiation.cc:240] Negotiation successful
0806 09:28:27.812137 (+ 612us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":642,"thread_start_us":394,"threads_started":1}
I20260806 09:28:27.835618 22999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:27.820278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:37651 (local address 127.20.113.2:47681)
0806 09:28:27.820925 (+ 647us) negotiation.cc:107] Waiting for socket to connect
0806 09:28:27.820940 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 09:28:27.821054 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:28:27.821420 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:28:27.821454 (+ 34us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:28:27.822085 (+ 631us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:28:27.822576 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:27.822598 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:27.823993 (+ 1395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:27.823998 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:27.825382 (+ 1384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:27.825403 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:27.826575 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:27.826580 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:27.826696 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:27.827414 (+ 718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:28:27.827431 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:28:27.828125 (+ 694us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:28:27.832811 (+ 4686us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:28:27.832814 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:28:27.832817 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:28:27.833274 (+ 457us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:28:27.833811 (+ 537us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:28:27.833814 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:28:27.833816 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:28:27.833940 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:28:27.834769 (+ 829us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:28:27.834772 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:28:27.834929 (+ 157us) client_negotiation.cc:769] Sending connection context
0806 09:28:27.835055 (+ 126us) client_negotiation.cc:240] Negotiation successful
0806 09:28:27.835283 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":515,"thread_start_us":361,"threads_started":1}
I20260806 09:28:27.835983 22716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:27.820187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.2:47681 (local address 127.20.113.62:37651)
0806 09:28:27.820398 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:27.820402 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:27.821018 (+ 616us) server_negotiation.cc:407] Connection header received
0806 09:28:27.821173 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:27.821181 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:27.821284 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:27.821416 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:28:27.822743 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:27.823840 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:27.825551 (+ 1711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:27.826383 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:27.826554 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:27.828367 (+ 1813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:28:27.828377 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:28:27.828393 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:28:27.828442 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:28:27.832665 (+ 4223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:27.833423 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:27.833432 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:27.833438 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:27.833646 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:27.834054 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:27.834062 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:27.834068 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:27.834646 (+ 578us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:28:27.834835 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:27.835206 (+ 371us) server_negotiation.cc:299] Negotiation successful
0806 09:28:27.835509 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 09:28:27.845659 22880 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.2:42551
I20260806 09:28:27.845681 23066 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.2:42551 every 8 connection(s)
I20260806 09:28:27.846844 22880 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786008495763054-20932-0/minicluster-data/ts-1/data/info.pb
I20260806 09:28:27.848920 23068 heartbeater.cc:347] Connected to a master server at 127.20.113.62:37651
I20260806 09:28:27.849156 23068 heartbeater.cc:464] Registering TS with master...
I20260806 09:28:27.849808 23068 heartbeater.cc:511] Master 127.20.113.62:37651 requested a full tablet report, sending...
I20260806 09:28:27.851125 22583 ts_manager.cc:194] Registered new tserver with Master: 7b0735292a6c448891ef04ecf4df52a2 (127.20.113.2:42551)
I20260806 09:28:27.851953 22583 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.113.2@KRBTEST.COM'} at 127.20.113.2:35253
I20260806 09:28:27.853654 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 22880
I20260806 09:28:27.853888 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786008495763054-20932-0/minicluster-data/ts-1/wal/instance
I20260806 09:28:27.854327 20932 external_mini_cluster.cc:1428] Setting key 627b49b33c599353238e9813b078656e
WARNING: no policy specified for kudu/127.20.113.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.113.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.113.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.3.keytab.
Entry for principal kudu/127.20.113.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.3.keytab.
I20260806 09:28:27.905596 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786008495763054-20932-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786008495763054-20932-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786008495763054-20932-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786008495763054-20932-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.113.3:0
--local_ip_for_outbound_sockets=127.20.113.3
--webserver_interface=127.20.113.3
--webserver_port=0
--tserver_master_addrs=127.20.113.62:37651
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.3.keytab
--principal=kudu/127.20.113.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:43381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:28:28.253123 23076 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:28:28.253418 23076 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:28:28.253517 23076 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:28:28.261260 23076 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:28:28.261430 23076 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:28:28.261590 23076 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 09:28:28.261740 23076 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.113.3
I20260806 09:28:28.274164 23076 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:43381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786008495763054-20932-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786008495763054-20932-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.3.keytab
--principal=kudu/127.20.113.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.113.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786008495763054-20932-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.113.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.20.113.62:37651
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.113.3
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786008495763054-20932-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:28:28.275592 23076 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:28:28.277108 23076 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:28:28.290356 23083 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:28:28.290251 23084 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:28:28.290315 23086 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:28:28.291158 23076 server_base.cc:1034] running on GCE node
I20260806 09:28:28.291765 23076 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:28:28.292728 23076 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:28:28.293946 23076 hybrid_clock.cc:648] HybridClock initialized: now 1786008508293874 us; error 56 us; skew 500 ppm
I20260806 09:28:28.294240 23076 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:28:28 dist-test-slave-45dx krb5kdc[22520](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008508, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:28:28.297638 23076 init.cc:377] Logged in from keytab as kudu/127.20.113.3@KRBTEST.COM (short username kudu)
I20260806 09:28:28.299110 23076 webserver.cc:460] Webserver started at http://127.20.113.3:45691/ using document root <none> and password file <none>
I20260806 09:28:28.299839 23076 fs_manager.cc:359] Metadata directory not provided
I20260806 09:28:28.299964 23076 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:28:28.300369 23076 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:28:28.303339 23076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786008495763054-20932-0/minicluster-data/ts-2/data/instance:
uuid: "ad71a077f186448fb2fd3100c4ac5c21"
format_stamp: "Formatted at 2026-08-06 09:28:28 on dist-test-slave-45dx"
server_key: "183b5cee8e4568f415388421db6f634a"
server_key_iv: "d2e46f297b7ee3c81e32e9b1dddbe7b6"
server_key_version: "encryptionkey@0"
I20260806 09:28:28.304150 23076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786008495763054-20932-0/minicluster-data/ts-2/wal/instance:
uuid: "ad71a077f186448fb2fd3100c4ac5c21"
format_stamp: "Formatted at 2026-08-06 09:28:28 on dist-test-slave-45dx"
server_key: "183b5cee8e4568f415388421db6f634a"
server_key_iv: "d2e46f297b7ee3c81e32e9b1dddbe7b6"
server_key_version: "encryptionkey@0"
I20260806 09:28:28.309221 23076 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260806 09:28:28.313704 23097 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:28:28.315289 23076 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.005s sys 0.000s
I20260806 09:28:28.315464 23076 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786008495763054-20932-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786008495763054-20932-0/minicluster-data/ts-2/wal
uuid: "ad71a077f186448fb2fd3100c4ac5c21"
format_stamp: "Formatted at 2026-08-06 09:28:28 on dist-test-slave-45dx"
server_key: "183b5cee8e4568f415388421db6f634a"
server_key_iv: "d2e46f297b7ee3c81e32e9b1dddbe7b6"
server_key_version: "encryptionkey@0"
I20260806 09:28:28.315707 23076 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786008495763054-20932-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786008495763054-20932-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786008495763054-20932-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:28:28.322956 22875 heartbeater.cc:503] Master 127.20.113.62:37651 was elected leader, sending a full tablet report...
I20260806 09:28:28.338881 23076 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:28:28.343731 23076 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:28:28.344065 23076 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:28:28.346884 23076 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:28:28.347002 23076 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:28:28.347175 23076 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:28:28.347249 23076 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 09:28:28 dist-test-slave-45dx krb5kdc[22520](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008508, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.3@KRBTEST.COM for kudu/127.20.113.62@KRBTEST.COM
I20260806 09:28:28.374941 23110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:28.347820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.3:56013 (local address 127.20.113.62:37651)
0806 09:28:28.350694 (+ 2874us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:28.350701 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:28.350727 (+ 26us) server_negotiation.cc:407] Connection header received
0806 09:28:28.353031 (+ 2304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:28.353037 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:28.353114 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:28.353183 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:28:28.356023 (+ 2840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:28.357457 (+ 1434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:28.360117 (+ 2660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:28.360760 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:28.360921 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:28.366170 (+ 5249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:28:28.366181 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:28:28.366187 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:28:28.366247 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:28:28.370794 (+ 4547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:28.372112 (+ 1318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:28.372120 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:28.372127 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:28.372379 (+ 252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:28.372892 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:28.372898 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:28.372902 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:28.373218 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:28:28.373382 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:28.374214 (+ 832us) server_negotiation.cc:299] Negotiation successful
0806 09:28:28.374456 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2727,"thread_start_us":2599,"threads_started":1}
I20260806 09:28:28.376458 23111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:28.348687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:37651 (local address 127.20.113.3:56013)
0806 09:28:28.349902 (+ 1215us) negotiation.cc:107] Waiting for socket to connect
0806 09:28:28.349988 (+ 86us) client_negotiation.cc:174] Beginning negotiation
0806 09:28:28.352553 (+ 2565us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:28:28.353843 (+ 1290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:28:28.353854 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:28:28.354824 (+ 970us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:28:28.355755 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:28.355777 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:28.357659 (+ 1882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:28.357667 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:28.359921 (+ 2254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:28.359939 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:28.361051 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:28.361063 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:28.361290 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:28.362223 (+ 933us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:28:28.362271 (+ 48us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:28:28.365885 (+ 3614us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:28:28.371034 (+ 5149us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:28:28.371047 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:28:28.371075 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:28:28.371950 (+ 875us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:28:28.372515 (+ 565us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:28:28.372520 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:28:28.372524 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:28:28.372765 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:28:28.373369 (+ 604us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:28:28.373377 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:28:28.373815 (+ 438us) client_negotiation.cc:769] Sending connection context
0806 09:28:28.374113 (+ 298us) client_negotiation.cc:240] Negotiation successful
0806 09:28:28.374719 (+ 606us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":783,"thread_start_us":440,"threads_started":1}
I20260806 09:28:28.400381 23076 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.3:44797
I20260806 09:28:28.400405 23267 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.3:44797 every 8 connection(s)
I20260806 09:28:28.401794 23076 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786008495763054-20932-0/minicluster-data/ts-2/data/info.pb
I20260806 09:28:28.404085 23269 heartbeater.cc:347] Connected to a master server at 127.20.113.62:37651
I20260806 09:28:28.404399 23269 heartbeater.cc:464] Registering TS with master...
I20260806 09:28:28.404891 23110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:28.384699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.3:47399 (local address 127.20.113.62:37651)
0806 09:28:28.384931 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:28.384941 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:28.388992 (+ 4051us) server_negotiation.cc:407] Connection header received
0806 09:28:28.389217 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:28.389221 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:28.389294 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:28.389375 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:28:28.391026 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:28.392046 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:28.394878 (+ 2832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:28.395441 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:28.395593 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:28.398419 (+ 2826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:28:28.398425 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:28:28.398452 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:28:28.398485 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:28:28.401552 (+ 3067us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:28.402326 (+ 774us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:28.402332 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:28.402336 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:28.402474 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:28.403204 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:28.403209 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:28.403212 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:28.403576 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:28:28.403748 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:28.404208 (+ 460us) server_negotiation.cc:299] Negotiation successful
0806 09:28:28.404495 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260806 09:28:28.404990 23230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:28.384724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:37651 (local address 127.20.113.3:47399)
0806 09:28:28.388874 (+ 4150us) negotiation.cc:107] Waiting for socket to connect
0806 09:28:28.388895 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 09:28:28.389066 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:28:28.389437 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:28:28.389444 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:28:28.390173 (+ 729us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:28:28.390791 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:28.390808 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:28.392214 (+ 1406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:28.392223 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:28.394693 (+ 2470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:28.394715 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:28.395886 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:28.395897 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:28.396123 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:28.396961 (+ 838us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:28:28.396990 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:28:28.398210 (+ 1220us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:28:28.401686 (+ 3476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:28:28.401692 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:28:28.401697 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:28:28.402196 (+ 499us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:28:28.402591 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:28:28.402596 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:28:28.402600 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:28:28.402791 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:28:28.403722 (+ 931us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:28:28.403726 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:28:28.403932 (+ 206us) client_negotiation.cc:769] Sending connection context
0806 09:28:28.404093 (+ 161us) client_negotiation.cc:240] Negotiation successful
0806 09:28:28.404498 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3964,"thread_start_us":3842,"threads_started":1}
I20260806 09:28:28.405457 23269 heartbeater.cc:511] Master 127.20.113.62:37651 requested a full tablet report, sending...
I20260806 09:28:28.406868 22583 ts_manager.cc:194] Registered new tserver with Master: ad71a077f186448fb2fd3100c4ac5c21 (127.20.113.3:44797)
I20260806 09:28:28.407982 22583 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.113.3@KRBTEST.COM'} at 127.20.113.3:56013
I20260806 09:28:28.409193 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 23076
I20260806 09:28:28.409387 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786008495763054-20932-0/minicluster-data/ts-2/wal/instance
I20260806 09:28:28.409842 20932 external_mini_cluster.cc:1428] Setting key 321176c4a46f42de3f12ae0bf1454960
I20260806 09:28:28.412705 20932 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 09:28:28 dist-test-slave-45dx krb5kdc[22520](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008508, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 09:28:28 dist-test-slave-45dx krb5kdc[22520](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008508, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.20.113.62@KRBTEST.COM
I20260806 09:28:28.468492 23110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:28.454805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52950 (local address 127.20.113.62:37651)
0806 09:28:28.455050 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:28.455057 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:28.455260 (+ 203us) server_negotiation.cc:407] Connection header received
0806 09:28:28.455431 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:28.455435 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:28.455501 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:28.455588 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:28:28.457067 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:28.457817 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:28.459043 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:28.459472 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:28.459571 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:28.463549 (+ 3978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:28:28.463554 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:28:28.463557 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:28:28.463590 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:28:28.466050 (+ 2460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:28.466619 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:28.466623 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:28.466625 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:28.466754 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:28.467153 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:28.467158 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:28.467160 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:28.467453 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:28:28.467670 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:28.467891 (+ 221us) server_negotiation.cc:299] Negotiation successful
0806 09:28:28.468060 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260806 09:28:28.472978 22583 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52950:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 09:28:28.480048 22583 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 09:28:28.516575 23288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:28.492637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58688 (local address 127.20.113.1:34967)
0806 09:28:28.506345 (+ 13708us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:28.506361 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:28.507786 (+ 1425us) server_negotiation.cc:407] Connection header received
0806 09:28:28.507920 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:28.507928 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:28.508211 (+ 283us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:28.508451 (+ 240us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:28:28.509473 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:28.511104 (+ 1631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:28.513619 (+ 2515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:28.514903 (+ 1284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:28.515048 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:28.515295 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:28.515991 (+ 696us) server_negotiation.cc:299] Negotiation successful
0806 09:28:28.516206 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13431,"thread_start_us":13700,"threads_started":1}
I20260806 09:28:28.516776 23291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:28.493259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32936 (local address 127.20.113.2:42551)
0806 09:28:28.507722 (+ 14463us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:28.507732 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:28.508865 (+ 1133us) server_negotiation.cc:407] Connection header received
0806 09:28:28.508978 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:28.508987 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:28.509233 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:28.509434 (+ 201us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:28:28.510432 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:28.511650 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:28.514269 (+ 2619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:28.515655 (+ 1386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:28.515805 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:28.516097 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:28.516278 (+ 181us) server_negotiation.cc:299] Negotiation successful
0806 09:28:28.516473 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14285,"thread_start_us":14118,"threads_started":1}
I20260806 09:28:28.517066 23292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:28.492643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.2:42551 (local address 127.0.0.1:32936)
0806 09:28:28.508572 (+ 15929us) negotiation.cc:107] Waiting for socket to connect
0806 09:28:28.508596 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 09:28:28.508723 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:28:28.509702 (+ 979us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:28:28.509707 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:28:28.509731 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:28:28.510204 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:28.510221 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:28.511785 (+ 1564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:28.511793 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:28.514093 (+ 2300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:28.514109 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:28.515911 (+ 1802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:28.515919 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:28.516077 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:28.516114 (+ 37us) client_negotiation.cc:769] Sending connection context
0806 09:28:28.516238 (+ 124us) client_negotiation.cc:240] Negotiation successful
0806 09:28:28.516477 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15774,"mutex_wait_us":50,"thread_start_us":15978,"threads_started":1}
I20260806 09:28:28.517547 23287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:28.492445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.1:34967 (local address 127.0.0.1:58688)
0806 09:28:28.507231 (+ 14786us) negotiation.cc:107] Waiting for socket to connect
0806 09:28:28.507328 (+ 97us) client_negotiation.cc:174] Beginning negotiation
0806 09:28:28.507547 (+ 219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:28:28.508720 (+ 1173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:28:28.508729 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:28:28.508761 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:28:28.509261 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:28.509277 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:28.511255 (+ 1978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:28.511260 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:28.513451 (+ 2191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:28.513464 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:28.515434 (+ 1970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:28.515441 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:28.515664 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:28.515734 (+ 70us) client_negotiation.cc:769] Sending connection context
0806 09:28:28.516820 (+ 1086us) client_negotiation.cc:240] Negotiation successful
0806 09:28:28.517038 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14533,"mutex_wait_us":40,"thread_start_us":14368,"threads_started":1}
I20260806 09:28:28.518038 23289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:28.494626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.3:44797 (local address 127.0.0.1:37104)
0806 09:28:28.508921 (+ 14295us) negotiation.cc:107] Waiting for socket to connect
0806 09:28:28.508935 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 09:28:28.509095 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:28:28.510118 (+ 1023us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:28:28.510122 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:28:28.510158 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:28:28.510568 (+ 410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:28.510579 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:28.512544 (+ 1965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:28.512550 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:28.514697 (+ 2147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:28.514709 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:28.517118 (+ 2409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:28.517126 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:28.517399 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:28.517433 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 09:28:28.517532 (+ 99us) client_negotiation.cc:240] Negotiation successful
0806 09:28:28.517661 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14168,"mutex_wait_us":1,"thread_start_us":13612,"threads_started":1}
I20260806 09:28:28.518157 23294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:28.495448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37104 (local address 127.20.113.3:44797)
0806 09:28:28.508406 (+ 12958us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:28.508416 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:28.509066 (+ 650us) server_negotiation.cc:407] Connection header received
0806 09:28:28.509279 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:28.509286 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:28.509526 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:28.509728 (+ 202us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:28:28.510780 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:28.512391 (+ 1611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:28.514895 (+ 2504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:28.516465 (+ 1570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:28.516598 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:28.517554 (+ 956us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:28.517623 (+ 69us) server_negotiation.cc:299] Negotiation successful
0806 09:28:28.517791 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12734,"thread_start_us":12534,"threads_started":1}
I20260806 09:28:28.521405 22968 tablet_service.cc:1467] Processing CreateTablet for tablet 4cc9bfad1748495c9f64a6eace386f6b (DEFAULT_TABLE table=test-table [id=15e96e6b2da4409f8ebe45bd9e7ff859]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:28:28.521387 22771 tablet_service.cc:1467] Processing CreateTablet for tablet 4cc9bfad1748495c9f64a6eace386f6b (DEFAULT_TABLE table=test-table [id=15e96e6b2da4409f8ebe45bd9e7ff859]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:28:28.521988 23166 tablet_service.cc:1467] Processing CreateTablet for tablet 4cc9bfad1748495c9f64a6eace386f6b (DEFAULT_TABLE table=test-table [id=15e96e6b2da4409f8ebe45bd9e7ff859]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:28:28.522815 22968 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4cc9bfad1748495c9f64a6eace386f6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:28:28.522898 22771 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4cc9bfad1748495c9f64a6eace386f6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:28:28.523329 23166 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4cc9bfad1748495c9f64a6eace386f6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:28:28.535377 23295 tablet_bootstrap.cc:492] T 4cc9bfad1748495c9f64a6eace386f6b P 7b0735292a6c448891ef04ecf4df52a2: Bootstrap starting.
I20260806 09:28:28.538779 23298 tablet_bootstrap.cc:492] T 4cc9bfad1748495c9f64a6eace386f6b P ad71a077f186448fb2fd3100c4ac5c21: Bootstrap starting.
I20260806 09:28:28.539883 23295 tablet_bootstrap.cc:654] T 4cc9bfad1748495c9f64a6eace386f6b P 7b0735292a6c448891ef04ecf4df52a2: Neither blocks nor log segments found. Creating new log.
I20260806 09:28:28.541221 23296 tablet_bootstrap.cc:492] T 4cc9bfad1748495c9f64a6eace386f6b P 63e98f6f427648d298613653395a2702: Bootstrap starting.
I20260806 09:28:28.541982 23295 log.cc:824] T 4cc9bfad1748495c9f64a6eace386f6b P 7b0735292a6c448891ef04ecf4df52a2: Log is configured to *not* fsync() on all Append() calls
I20260806 09:28:28.542970 23298 tablet_bootstrap.cc:654] T 4cc9bfad1748495c9f64a6eace386f6b P ad71a077f186448fb2fd3100c4ac5c21: Neither blocks nor log segments found. Creating new log.
I20260806 09:28:28.544260 23298 log.cc:824] T 4cc9bfad1748495c9f64a6eace386f6b P ad71a077f186448fb2fd3100c4ac5c21: Log is configured to *not* fsync() on all Append() calls
I20260806 09:28:28.544764 23296 tablet_bootstrap.cc:654] T 4cc9bfad1748495c9f64a6eace386f6b P 63e98f6f427648d298613653395a2702: Neither blocks nor log segments found. Creating new log.
I20260806 09:28:28.545501 23295 tablet_bootstrap.cc:492] T 4cc9bfad1748495c9f64a6eace386f6b P 7b0735292a6c448891ef04ecf4df52a2: No bootstrap required, opened a new log
I20260806 09:28:28.545819 23295 ts_tablet_manager.cc:1397] T 4cc9bfad1748495c9f64a6eace386f6b P 7b0735292a6c448891ef04ecf4df52a2: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.003s
I20260806 09:28:28.546124 23296 log.cc:824] T 4cc9bfad1748495c9f64a6eace386f6b P 63e98f6f427648d298613653395a2702: Log is configured to *not* fsync() on all Append() calls
I20260806 09:28:28.547530 23298 tablet_bootstrap.cc:492] T 4cc9bfad1748495c9f64a6eace386f6b P ad71a077f186448fb2fd3100c4ac5c21: No bootstrap required, opened a new log
I20260806 09:28:28.547809 23298 ts_tablet_manager.cc:1397] T 4cc9bfad1748495c9f64a6eace386f6b P ad71a077f186448fb2fd3100c4ac5c21: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.004s
I20260806 09:28:28.549175 23296 tablet_bootstrap.cc:492] T 4cc9bfad1748495c9f64a6eace386f6b P 63e98f6f427648d298613653395a2702: No bootstrap required, opened a new log
I20260806 09:28:28.549437 23296 ts_tablet_manager.cc:1397] T 4cc9bfad1748495c9f64a6eace386f6b P 63e98f6f427648d298613653395a2702: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.001s
I20260806 09:28:28.556856 23296 raft_consensus.cc:348] T 4cc9bfad1748495c9f64a6eace386f6b P 63e98f6f427648d298613653395a2702 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63e98f6f427648d298613653395a2702" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 34967 } } peers { permanent_uuid: "7b0735292a6c448891ef04ecf4df52a2" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 42551 } } peers { permanent_uuid: "ad71a077f186448fb2fd3100c4ac5c21" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 44797 } }
I20260806 09:28:28.557262 23296 raft_consensus.cc:374] T 4cc9bfad1748495c9f64a6eace386f6b P 63e98f6f427648d298613653395a2702 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:28:28.556978 23295 raft_consensus.cc:348] T 4cc9bfad1748495c9f64a6eace386f6b P 7b0735292a6c448891ef04ecf4df52a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63e98f6f427648d298613653395a2702" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 34967 } } peers { permanent_uuid: "7b0735292a6c448891ef04ecf4df52a2" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 42551 } } peers { permanent_uuid: "ad71a077f186448fb2fd3100c4ac5c21" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 44797 } }
I20260806 09:28:28.557413 23295 raft_consensus.cc:374] T 4cc9bfad1748495c9f64a6eace386f6b P 7b0735292a6c448891ef04ecf4df52a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:28:28.557421 23296 raft_consensus.cc:729] T 4cc9bfad1748495c9f64a6eace386f6b P 63e98f6f427648d298613653395a2702 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63e98f6f427648d298613653395a2702, State: Initialized, Role: FOLLOWER
I20260806 09:28:28.557521 23295 raft_consensus.cc:729] T 4cc9bfad1748495c9f64a6eace386f6b P 7b0735292a6c448891ef04ecf4df52a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b0735292a6c448891ef04ecf4df52a2, State: Initialized, Role: FOLLOWER
I20260806 09:28:28.558121 23296 consensus_queue.cc:260] T 4cc9bfad1748495c9f64a6eace386f6b P 63e98f6f427648d298613653395a2702 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63e98f6f427648d298613653395a2702" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 34967 } } peers { permanent_uuid: "7b0735292a6c448891ef04ecf4df52a2" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 42551 } } peers { permanent_uuid: "ad71a077f186448fb2fd3100c4ac5c21" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 44797 } }
I20260806 09:28:28.558234 23295 consensus_queue.cc:260] T 4cc9bfad1748495c9f64a6eace386f6b P 7b0735292a6c448891ef04ecf4df52a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63e98f6f427648d298613653395a2702" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 34967 } } peers { permanent_uuid: "7b0735292a6c448891ef04ecf4df52a2" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 42551 } } peers { permanent_uuid: "ad71a077f186448fb2fd3100c4ac5c21" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 44797 } }
I20260806 09:28:28.558972 23298 raft_consensus.cc:348] T 4cc9bfad1748495c9f64a6eace386f6b P ad71a077f186448fb2fd3100c4ac5c21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63e98f6f427648d298613653395a2702" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 34967 } } peers { permanent_uuid: "7b0735292a6c448891ef04ecf4df52a2" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 42551 } } peers { permanent_uuid: "ad71a077f186448fb2fd3100c4ac5c21" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 44797 } }
I20260806 09:28:28.559608 23298 raft_consensus.cc:374] T 4cc9bfad1748495c9f64a6eace386f6b P ad71a077f186448fb2fd3100c4ac5c21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:28:28.559787 23298 raft_consensus.cc:729] T 4cc9bfad1748495c9f64a6eace386f6b P ad71a077f186448fb2fd3100c4ac5c21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad71a077f186448fb2fd3100c4ac5c21, State: Initialized, Role: FOLLOWER
I20260806 09:28:28.559978 23296 ts_tablet_manager.cc:1428] T 4cc9bfad1748495c9f64a6eace386f6b P 63e98f6f427648d298613653395a2702: Time spent starting tablet: real 0.010s user 0.003s sys 0.007s
I20260806 09:28:28.560681 23298 consensus_queue.cc:260] T 4cc9bfad1748495c9f64a6eace386f6b P ad71a077f186448fb2fd3100c4ac5c21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63e98f6f427648d298613653395a2702" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 34967 } } peers { permanent_uuid: "7b0735292a6c448891ef04ecf4df52a2" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 42551 } } peers { permanent_uuid: "ad71a077f186448fb2fd3100c4ac5c21" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 44797 } }
I20260806 09:28:28.562039 23269 heartbeater.cc:503] Master 127.20.113.62:37651 was elected leader, sending a full tablet report...
I20260806 09:28:28.562867 23298 ts_tablet_manager.cc:1428] T 4cc9bfad1748495c9f64a6eace386f6b P ad71a077f186448fb2fd3100c4ac5c21: Time spent starting tablet: real 0.015s user 0.014s sys 0.000s
W20260806 09:28:28.565231 22877 tablet.cc:2369] T 4cc9bfad1748495c9f64a6eace386f6b P 63e98f6f427648d298613653395a2702: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 09:28:28.565810 23295 ts_tablet_manager.cc:1428] T 4cc9bfad1748495c9f64a6eace386f6b P 7b0735292a6c448891ef04ecf4df52a2: Time spent starting tablet: real 0.020s user 0.013s sys 0.006s
I20260806 09:28:28.565846 23068 heartbeater.cc:503] Master 127.20.113.62:37651 was elected leader, sending a full tablet report...
W20260806 09:28:28.598999 23071 tablet.cc:2369] T 4cc9bfad1748495c9f64a6eace386f6b P 7b0735292a6c448891ef04ecf4df52a2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 09:28:28.653676 23270 tablet.cc:2369] T 4cc9bfad1748495c9f64a6eace386f6b P ad71a077f186448fb2fd3100c4ac5c21: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 09:28:28.726910 23302 raft_consensus.cc:482] T 4cc9bfad1748495c9f64a6eace386f6b P 63e98f6f427648d298613653395a2702 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 09:28:28.727296 23302 raft_consensus.cc:504] T 4cc9bfad1748495c9f64a6eace386f6b P 63e98f6f427648d298613653395a2702 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63e98f6f427648d298613653395a2702" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 34967 } } peers { permanent_uuid: "7b0735292a6c448891ef04ecf4df52a2" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 42551 } } peers { permanent_uuid: "ad71a077f186448fb2fd3100c4ac5c21" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 44797 } }
I20260806 09:28:28.729242 23302 leader_election.cc:288] T 4cc9bfad1748495c9f64a6eace386f6b P 63e98f6f427648d298613653395a2702 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7b0735292a6c448891ef04ecf4df52a2 (127.20.113.2:42551), ad71a077f186448fb2fd3100c4ac5c21 (127.20.113.3:44797)
I20260806 09:28:28.737099 23291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:28.729587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.1:46301 (local address 127.20.113.2:42551)
0806 09:28:28.729865 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:28.729874 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:28.730837 (+ 963us) server_negotiation.cc:407] Connection header received
0806 09:28:28.731067 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:28.731075 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:28.731160 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:28.731232 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:28:28.732034 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:28.733044 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:28.735361 (+ 2317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:28.736123 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:28.736342 (+ 219us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:28.736429 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:28.736705 (+ 276us) server_negotiation.cc:299] Negotiation successful
0806 09:28:28.736832 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260806 09:28:28.737285 23306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:28.729622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.2:42551 (local address 127.20.113.1:46301)
0806 09:28:28.730709 (+ 1087us) negotiation.cc:107] Waiting for socket to connect
0806 09:28:28.730725 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 09:28:28.730901 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:28:28.731284 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:28:28.731291 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:28:28.731322 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:28:28.731862 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:28.731877 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:28.733169 (+ 1292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:28.733177 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:28.735212 (+ 2035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:28.735225 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:28.736357 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:28.736364 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:28.736522 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:28.736565 (+ 43us) client_negotiation.cc:769] Sending connection context
0806 09:28:28.736677 (+ 112us) client_negotiation.cc:240] Negotiation successful
0806 09:28:28.736807 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":942,"mutex_wait_us":6,"spinlock_wait_cycles":34048,"thread_start_us":605,"threads_started":1}
I20260806 09:28:28.737342 23294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:28.729993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.1:58781 (local address 127.20.113.3:44797)
0806 09:28:28.730211 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:28.730219 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:28.730907 (+ 688us) server_negotiation.cc:407] Connection header received
0806 09:28:28.731147 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:28.731151 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:28.731239 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:28.731338 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:28:28.732101 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:28.733103 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:28.735467 (+ 2364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:28.736396 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:28.736600 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:28.736672 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:28.736950 (+ 278us) server_negotiation.cc:299] Negotiation successful
0806 09:28:28.737079 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 09:28:28.737504 23311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:28.729750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.3:44797 (local address 127.20.113.1:58781)
0806 09:28:28.730762 (+ 1012us) negotiation.cc:107] Waiting for socket to connect
0806 09:28:28.730778 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 09:28:28.730939 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:28:28.731371 (+ 432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:28:28.731376 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:28:28.731410 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:28:28.731917 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:28.731942 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:28.733255 (+ 1313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:28.733260 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:28.735300 (+ 2040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:28.735318 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:28.736623 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:28.736630 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:28.736794 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:28.736818 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 09:28:28.736919 (+ 101us) client_negotiation.cc:240] Negotiation successful
0806 09:28:28.737043 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":905,"mutex_wait_us":25,"thread_start_us":612,"threads_started":1}
I20260806 09:28:28.738605 23005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4cc9bfad1748495c9f64a6eace386f6b" candidate_uuid: "63e98f6f427648d298613653395a2702" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b0735292a6c448891ef04ecf4df52a2" is_pre_election: true
I20260806 09:28:28.738605 23200 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4cc9bfad1748495c9f64a6eace386f6b" candidate_uuid: "63e98f6f427648d298613653395a2702" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad71a077f186448fb2fd3100c4ac5c21" is_pre_election: true
I20260806 09:28:28.739166 23200 raft_consensus.cc:2454] T 4cc9bfad1748495c9f64a6eace386f6b P ad71a077f186448fb2fd3100c4ac5c21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 63e98f6f427648d298613653395a2702 in term 0.
I20260806 09:28:28.739200 23005 raft_consensus.cc:2454] T 4cc9bfad1748495c9f64a6eace386f6b P 7b0735292a6c448891ef04ecf4df52a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 63e98f6f427648d298613653395a2702 in term 0.
I20260806 09:28:28.739926 22705 leader_election.cc:302] T 4cc9bfad1748495c9f64a6eace386f6b P 63e98f6f427648d298613653395a2702 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 63e98f6f427648d298613653395a2702, 7b0735292a6c448891ef04ecf4df52a2, ad71a077f186448fb2fd3100c4ac5c21; no voters:
I20260806 09:28:28.740368 23302 raft_consensus.cc:2781] T 4cc9bfad1748495c9f64a6eace386f6b P 63e98f6f427648d298613653395a2702 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 09:28:28.740540 23302 raft_consensus.cc:482] T 4cc9bfad1748495c9f64a6eace386f6b P 63e98f6f427648d298613653395a2702 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 09:28:28.740655 23302 raft_consensus.cc:3037] T 4cc9bfad1748495c9f64a6eace386f6b P 63e98f6f427648d298613653395a2702 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:28:28.743098 23302 raft_consensus.cc:504] T 4cc9bfad1748495c9f64a6eace386f6b P 63e98f6f427648d298613653395a2702 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63e98f6f427648d298613653395a2702" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 34967 } } peers { permanent_uuid: "7b0735292a6c448891ef04ecf4df52a2" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 42551 } } peers { permanent_uuid: "ad71a077f186448fb2fd3100c4ac5c21" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 44797 } }
I20260806 09:28:28.743748 23302 leader_election.cc:288] T 4cc9bfad1748495c9f64a6eace386f6b P 63e98f6f427648d298613653395a2702 [CANDIDATE]: Term 1 election: Requested vote from peers 7b0735292a6c448891ef04ecf4df52a2 (127.20.113.2:42551), ad71a077f186448fb2fd3100c4ac5c21 (127.20.113.3:44797)
I20260806 09:28:28.744067 23005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4cc9bfad1748495c9f64a6eace386f6b" candidate_uuid: "63e98f6f427648d298613653395a2702" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b0735292a6c448891ef04ecf4df52a2"
I20260806 09:28:28.744269 23005 raft_consensus.cc:3037] T 4cc9bfad1748495c9f64a6eace386f6b P 7b0735292a6c448891ef04ecf4df52a2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:28:28.744273 23200 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4cc9bfad1748495c9f64a6eace386f6b" candidate_uuid: "63e98f6f427648d298613653395a2702" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad71a077f186448fb2fd3100c4ac5c21"
I20260806 09:28:28.744510 23200 raft_consensus.cc:3037] T 4cc9bfad1748495c9f64a6eace386f6b P ad71a077f186448fb2fd3100c4ac5c21 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:28:28.746567 23005 raft_consensus.cc:2454] T 4cc9bfad1748495c9f64a6eace386f6b P 7b0735292a6c448891ef04ecf4df52a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 63e98f6f427648d298613653395a2702 in term 1.
I20260806 09:28:28.746927 23200 raft_consensus.cc:2454] T 4cc9bfad1748495c9f64a6eace386f6b P ad71a077f186448fb2fd3100c4ac5c21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 63e98f6f427648d298613653395a2702 in term 1.
I20260806 09:28:28.746932 22705 leader_election.cc:302] T 4cc9bfad1748495c9f64a6eace386f6b P 63e98f6f427648d298613653395a2702 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 63e98f6f427648d298613653395a2702, 7b0735292a6c448891ef04ecf4df52a2; no voters:
I20260806 09:28:28.747352 23302 raft_consensus.cc:2781] T 4cc9bfad1748495c9f64a6eace386f6b P 63e98f6f427648d298613653395a2702 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:28:28.747855 23302 raft_consensus.cc:686] T 4cc9bfad1748495c9f64a6eace386f6b P 63e98f6f427648d298613653395a2702 [term 1 LEADER]: Becoming Leader. State: Replica: 63e98f6f427648d298613653395a2702, State: Running, Role: LEADER
I20260806 09:28:28.748361 23302 consensus_queue.cc:237] T 4cc9bfad1748495c9f64a6eace386f6b P 63e98f6f427648d298613653395a2702 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63e98f6f427648d298613653395a2702" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 34967 } } peers { permanent_uuid: "7b0735292a6c448891ef04ecf4df52a2" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 42551 } } peers { permanent_uuid: "ad71a077f186448fb2fd3100c4ac5c21" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 44797 } }
I20260806 09:28:28.753355 22581 catalog_manager.cc:5495] T 4cc9bfad1748495c9f64a6eace386f6b P 63e98f6f427648d298613653395a2702 reported cstate change: term changed from 0 to 1, leader changed from <none> to 63e98f6f427648d298613653395a2702 (127.20.113.1). New cstate: current_term: 1 leader_uuid: "63e98f6f427648d298613653395a2702" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63e98f6f427648d298613653395a2702" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 34967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7b0735292a6c448891ef04ecf4df52a2" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 42551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ad71a077f186448fb2fd3100c4ac5c21" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 44797 } health_report { overall_health: UNKNOWN } } }
I20260806 09:28:28.811915 23288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:28.806284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58690 (local address 127.20.113.1:34967)
0806 09:28:28.806528 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:28.806535 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:28.806560 (+ 25us) server_negotiation.cc:407] Connection header received
0806 09:28:28.806849 (+ 289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:28.806855 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:28.806962 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:28.807140 (+ 178us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 09:28:28.807546 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:28.808258 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:28.809674 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:28.810104 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:28.810256 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:28.810761 (+ 505us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 09:28:28.811265 (+ 504us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 09:28:28.811431 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:28.811559 (+ 128us) server_negotiation.cc:299] Negotiation successful
0806 09:28:28.811644 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 09:28:28.820896 23200 raft_consensus.cc:1261] T 4cc9bfad1748495c9f64a6eace386f6b P ad71a077f186448fb2fd3100c4ac5c21 [term 1 FOLLOWER]: Refusing update from remote peer 63e98f6f427648d298613653395a2702: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 09:28:28.820899 23005 raft_consensus.cc:1261] T 4cc9bfad1748495c9f64a6eace386f6b P 7b0735292a6c448891ef04ecf4df52a2 [term 1 FOLLOWER]: Refusing update from remote peer 63e98f6f427648d298613653395a2702: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 09:28:28.821944 23312 consensus_queue.cc:1035] T 4cc9bfad1748495c9f64a6eace386f6b P 63e98f6f427648d298613653395a2702 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7b0735292a6c448891ef04ecf4df52a2" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 42551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 09:28:28.822296 23302 consensus_queue.cc:1035] T 4cc9bfad1748495c9f64a6eace386f6b P 63e98f6f427648d298613653395a2702 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad71a077f186448fb2fd3100c4ac5c21" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 44797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 09:28:28.857460 22583 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52950:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 09:28:28.857766 22583 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52950:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 09:28:28.861450 22583 catalog_manager.cc:5782] T 00000000000000000000000000000000 P aefb46d3774747bca7b9baf9042e72a3: Sending DeleteTablet for 3 replicas of tablet 4cc9bfad1748495c9f64a6eace386f6b
I20260806 09:28:28.863530 22771 tablet_service.cc:1514] Processing DeleteTablet for tablet 4cc9bfad1748495c9f64a6eace386f6b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:28:28 UTC) from {username='kudu', principal='kudu/127.20.113.62@KRBTEST.COM'} at 127.0.0.1:58688
I20260806 09:28:28.865103 22968 tablet_service.cc:1514] Processing DeleteTablet for tablet 4cc9bfad1748495c9f64a6eace386f6b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:28:28 UTC) from {username='kudu', principal='kudu/127.20.113.62@KRBTEST.COM'} at 127.0.0.1:32936
I20260806 09:28:28.865571 23166 tablet_service.cc:1514] Processing DeleteTablet for tablet 4cc9bfad1748495c9f64a6eace386f6b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:28:28 UTC) from {username='kudu', principal='kudu/127.20.113.62@KRBTEST.COM'} at 127.0.0.1:37104
I20260806 09:28:28.866401 23332 tablet_replica.cc:330] T 4cc9bfad1748495c9f64a6eace386f6b P 63e98f6f427648d298613653395a2702: stopping tablet replica
I20260806 09:28:28.866945 23332 raft_consensus.cc:2229] T 4cc9bfad1748495c9f64a6eace386f6b P 63e98f6f427648d298613653395a2702 [term 1 LEADER]: Raft consensus shutting down.
I20260806 09:28:28.867856 23332 raft_consensus.cc:2258] T 4cc9bfad1748495c9f64a6eace386f6b P 63e98f6f427648d298613653395a2702 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 09:28:28.870398 23332 ts_tablet_manager.cc:1905] T 4cc9bfad1748495c9f64a6eace386f6b P 63e98f6f427648d298613653395a2702: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 09:28:28.870932 22583 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52950:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 09:28:28.871651 22583 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 09:28:28.879397 23334 tablet_replica.cc:330] T 4cc9bfad1748495c9f64a6eace386f6b P 7b0735292a6c448891ef04ecf4df52a2: stopping tablet replica
I20260806 09:28:28.879997 23334 raft_consensus.cc:2229] T 4cc9bfad1748495c9f64a6eace386f6b P 7b0735292a6c448891ef04ecf4df52a2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 09:28:28.880633 23334 raft_consensus.cc:2258] T 4cc9bfad1748495c9f64a6eace386f6b P 7b0735292a6c448891ef04ecf4df52a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 09:28:28.881783 23165 tablet_service.cc:1467] Processing CreateTablet for tablet 2d7871caf907422d86dfac7cfe4b9561 (DEFAULT_TABLE table=test-table [id=6a2e08afffbc4a74a10a0758bb70a64b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:28:28.882236 23333 tablet_replica.cc:330] T 4cc9bfad1748495c9f64a6eace386f6b P ad71a077f186448fb2fd3100c4ac5c21: stopping tablet replica
I20260806 09:28:28.882311 23165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d7871caf907422d86dfac7cfe4b9561. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:28:28.882218 22771 tablet_service.cc:1467] Processing CreateTablet for tablet 2d7871caf907422d86dfac7cfe4b9561 (DEFAULT_TABLE table=test-table [id=6a2e08afffbc4a74a10a0758bb70a64b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:28:28.882679 22771 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d7871caf907422d86dfac7cfe4b9561. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:28:28.882675 23333 raft_consensus.cc:2229] T 4cc9bfad1748495c9f64a6eace386f6b P ad71a077f186448fb2fd3100c4ac5c21 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 09:28:28.883217 23333 raft_consensus.cc:2258] T 4cc9bfad1748495c9f64a6eace386f6b P ad71a077f186448fb2fd3100c4ac5c21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 09:28:28.886325 23332 ts_tablet_manager.cc:1918] T 4cc9bfad1748495c9f64a6eace386f6b P 63e98f6f427648d298613653395a2702: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 09:28:28.886526 23332 log.cc:1196] T 4cc9bfad1748495c9f64a6eace386f6b P 63e98f6f427648d298613653395a2702: Deleting WAL directory at /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786008495763054-20932-0/minicluster-data/ts-0/wal/wals/4cc9bfad1748495c9f64a6eace386f6b
I20260806 09:28:28.887108 23332 ts_tablet_manager.cc:1939] T 4cc9bfad1748495c9f64a6eace386f6b P 63e98f6f427648d298613653395a2702: Deleting consensus metadata
I20260806 09:28:28.889454 23298 tablet_bootstrap.cc:492] T 2d7871caf907422d86dfac7cfe4b9561 P ad71a077f186448fb2fd3100c4ac5c21: Bootstrap starting.
I20260806 09:28:28.890120 22562 catalog_manager.cc:4841] TS 63e98f6f427648d298613653395a2702 (127.20.113.1:34967): tablet 4cc9bfad1748495c9f64a6eace386f6b (table test-table [id=15e96e6b2da4409f8ebe45bd9e7ff859]) successfully deleted
I20260806 09:28:28.890802 22968 tablet_service.cc:1467] Processing CreateTablet for tablet 2d7871caf907422d86dfac7cfe4b9561 (DEFAULT_TABLE table=test-table [id=6a2e08afffbc4a74a10a0758bb70a64b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:28:28.891179 23298 tablet_bootstrap.cc:654] T 2d7871caf907422d86dfac7cfe4b9561 P ad71a077f186448fb2fd3100c4ac5c21: Neither blocks nor log segments found. Creating new log.
I20260806 09:28:28.891311 22968 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d7871caf907422d86dfac7cfe4b9561. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:28:28.892019 23334 ts_tablet_manager.cc:1905] T 4cc9bfad1748495c9f64a6eace386f6b P 7b0735292a6c448891ef04ecf4df52a2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 09:28:28.892976 23296 tablet_bootstrap.cc:492] T 2d7871caf907422d86dfac7cfe4b9561 P 63e98f6f427648d298613653395a2702: Bootstrap starting.
I20260806 09:28:28.894711 23296 tablet_bootstrap.cc:654] T 2d7871caf907422d86dfac7cfe4b9561 P 63e98f6f427648d298613653395a2702: Neither blocks nor log segments found. Creating new log.
I20260806 09:28:28.898097 23333 ts_tablet_manager.cc:1905] T 4cc9bfad1748495c9f64a6eace386f6b P ad71a077f186448fb2fd3100c4ac5c21: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 09:28:28.901120 23298 tablet_bootstrap.cc:492] T 2d7871caf907422d86dfac7cfe4b9561 P ad71a077f186448fb2fd3100c4ac5c21: No bootstrap required, opened a new log
I20260806 09:28:28.901340 23298 ts_tablet_manager.cc:1397] T 2d7871caf907422d86dfac7cfe4b9561 P ad71a077f186448fb2fd3100c4ac5c21: Time spent bootstrapping tablet: real 0.012s user 0.010s sys 0.000s
I20260806 09:28:28.902406 23298 raft_consensus.cc:348] T 2d7871caf907422d86dfac7cfe4b9561 P ad71a077f186448fb2fd3100c4ac5c21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63e98f6f427648d298613653395a2702" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 34967 } } peers { permanent_uuid: "7b0735292a6c448891ef04ecf4df52a2" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 42551 } } peers { permanent_uuid: "ad71a077f186448fb2fd3100c4ac5c21" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 44797 } }
I20260806 09:28:28.902750 23298 raft_consensus.cc:374] T 2d7871caf907422d86dfac7cfe4b9561 P ad71a077f186448fb2fd3100c4ac5c21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:28:28.902915 23298 raft_consensus.cc:729] T 2d7871caf907422d86dfac7cfe4b9561 P ad71a077f186448fb2fd3100c4ac5c21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad71a077f186448fb2fd3100c4ac5c21, State: Initialized, Role: FOLLOWER
I20260806 09:28:28.903353 23298 consensus_queue.cc:260] T 2d7871caf907422d86dfac7cfe4b9561 P ad71a077f186448fb2fd3100c4ac5c21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63e98f6f427648d298613653395a2702" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 34967 } } peers { permanent_uuid: "7b0735292a6c448891ef04ecf4df52a2" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 42551 } } peers { permanent_uuid: "ad71a077f186448fb2fd3100c4ac5c21" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 44797 } }
I20260806 09:28:28.904402 23295 tablet_bootstrap.cc:492] T 2d7871caf907422d86dfac7cfe4b9561 P 7b0735292a6c448891ef04ecf4df52a2: Bootstrap starting.
I20260806 09:28:28.903404 23334 ts_tablet_manager.cc:1918] T 4cc9bfad1748495c9f64a6eace386f6b P 7b0735292a6c448891ef04ecf4df52a2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 09:28:28.904685 23334 log.cc:1196] T 4cc9bfad1748495c9f64a6eace386f6b P 7b0735292a6c448891ef04ecf4df52a2: Deleting WAL directory at /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786008495763054-20932-0/minicluster-data/ts-1/wal/wals/4cc9bfad1748495c9f64a6eace386f6b
I20260806 09:28:28.905618 23334 ts_tablet_manager.cc:1939] T 4cc9bfad1748495c9f64a6eace386f6b P 7b0735292a6c448891ef04ecf4df52a2: Deleting consensus metadata
I20260806 09:28:28.905954 23296 tablet_bootstrap.cc:492] T 2d7871caf907422d86dfac7cfe4b9561 P 63e98f6f427648d298613653395a2702: No bootstrap required, opened a new log
I20260806 09:28:28.906109 23296 ts_tablet_manager.cc:1397] T 2d7871caf907422d86dfac7cfe4b9561 P 63e98f6f427648d298613653395a2702: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.000s
I20260806 09:28:28.906265 23295 tablet_bootstrap.cc:654] T 2d7871caf907422d86dfac7cfe4b9561 P 7b0735292a6c448891ef04ecf4df52a2: Neither blocks nor log segments found. Creating new log.
I20260806 09:28:28.907017 23296 raft_consensus.cc:348] T 2d7871caf907422d86dfac7cfe4b9561 P 63e98f6f427648d298613653395a2702 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63e98f6f427648d298613653395a2702" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 34967 } } peers { permanent_uuid: "7b0735292a6c448891ef04ecf4df52a2" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 42551 } } peers { permanent_uuid: "ad71a077f186448fb2fd3100c4ac5c21" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 44797 } }
I20260806 09:28:28.905570 23333 ts_tablet_manager.cc:1918] T 4cc9bfad1748495c9f64a6eace386f6b P ad71a077f186448fb2fd3100c4ac5c21: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 09:28:28.907529 23333 log.cc:1196] T 4cc9bfad1748495c9f64a6eace386f6b P ad71a077f186448fb2fd3100c4ac5c21: Deleting WAL directory at /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786008495763054-20932-0/minicluster-data/ts-2/wal/wals/4cc9bfad1748495c9f64a6eace386f6b
I20260806 09:28:28.907543 22560 catalog_manager.cc:4841] TS 7b0735292a6c448891ef04ecf4df52a2 (127.20.113.2:42551): tablet 4cc9bfad1748495c9f64a6eace386f6b (table test-table [id=15e96e6b2da4409f8ebe45bd9e7ff859]) successfully deleted
I20260806 09:28:28.907754 23296 raft_consensus.cc:374] T 2d7871caf907422d86dfac7cfe4b9561 P 63e98f6f427648d298613653395a2702 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:28:28.907806 23298 ts_tablet_manager.cc:1428] T 2d7871caf907422d86dfac7cfe4b9561 P ad71a077f186448fb2fd3100c4ac5c21: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260806 09:28:28.907872 23296 raft_consensus.cc:729] T 2d7871caf907422d86dfac7cfe4b9561 P 63e98f6f427648d298613653395a2702 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63e98f6f427648d298613653395a2702, State: Initialized, Role: FOLLOWER
I20260806 09:28:28.908179 23333 ts_tablet_manager.cc:1939] T 4cc9bfad1748495c9f64a6eace386f6b P ad71a077f186448fb2fd3100c4ac5c21: Deleting consensus metadata
I20260806 09:28:28.908180 23296 consensus_queue.cc:260] T 2d7871caf907422d86dfac7cfe4b9561 P 63e98f6f427648d298613653395a2702 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63e98f6f427648d298613653395a2702" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 34967 } } peers { permanent_uuid: "7b0735292a6c448891ef04ecf4df52a2" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 42551 } } peers { permanent_uuid: "ad71a077f186448fb2fd3100c4ac5c21" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 44797 } }
I20260806 09:28:28.909003 23296 ts_tablet_manager.cc:1428] T 2d7871caf907422d86dfac7cfe4b9561 P 63e98f6f427648d298613653395a2702: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 09:28:28.910264 22564 catalog_manager.cc:4841] TS ad71a077f186448fb2fd3100c4ac5c21 (127.20.113.3:44797): tablet 4cc9bfad1748495c9f64a6eace386f6b (table test-table [id=15e96e6b2da4409f8ebe45bd9e7ff859]) successfully deleted
I20260806 09:28:28.911757 23295 tablet_bootstrap.cc:492] T 2d7871caf907422d86dfac7cfe4b9561 P 7b0735292a6c448891ef04ecf4df52a2: No bootstrap required, opened a new log
I20260806 09:28:28.911983 23295 ts_tablet_manager.cc:1397] T 2d7871caf907422d86dfac7cfe4b9561 P 7b0735292a6c448891ef04ecf4df52a2: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.003s
I20260806 09:28:28.912954 23295 raft_consensus.cc:348] T 2d7871caf907422d86dfac7cfe4b9561 P 7b0735292a6c448891ef04ecf4df52a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63e98f6f427648d298613653395a2702" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 34967 } } peers { permanent_uuid: "7b0735292a6c448891ef04ecf4df52a2" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 42551 } } peers { permanent_uuid: "ad71a077f186448fb2fd3100c4ac5c21" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 44797 } }
I20260806 09:28:28.913288 23295 raft_consensus.cc:374] T 2d7871caf907422d86dfac7cfe4b9561 P 7b0735292a6c448891ef04ecf4df52a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:28:28.913455 23295 raft_consensus.cc:729] T 2d7871caf907422d86dfac7cfe4b9561 P 7b0735292a6c448891ef04ecf4df52a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b0735292a6c448891ef04ecf4df52a2, State: Initialized, Role: FOLLOWER
I20260806 09:28:28.913781 23295 consensus_queue.cc:260] T 2d7871caf907422d86dfac7cfe4b9561 P 7b0735292a6c448891ef04ecf4df52a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63e98f6f427648d298613653395a2702" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 34967 } } peers { permanent_uuid: "7b0735292a6c448891ef04ecf4df52a2" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 42551 } } peers { permanent_uuid: "ad71a077f186448fb2fd3100c4ac5c21" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 44797 } }
I20260806 09:28:28.914335 23295 ts_tablet_manager.cc:1428] T 2d7871caf907422d86dfac7cfe4b9561 P 7b0735292a6c448891ef04ecf4df52a2: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260806 09:28:28.995528 23303 raft_consensus.cc:482] T 2d7871caf907422d86dfac7cfe4b9561 P 7b0735292a6c448891ef04ecf4df52a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 09:28:28.995828 23303 raft_consensus.cc:504] T 2d7871caf907422d86dfac7cfe4b9561 P 7b0735292a6c448891ef04ecf4df52a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63e98f6f427648d298613653395a2702" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 34967 } } peers { permanent_uuid: "7b0735292a6c448891ef04ecf4df52a2" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 42551 } } peers { permanent_uuid: "ad71a077f186448fb2fd3100c4ac5c21" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 44797 } }
I20260806 09:28:28.997769 23303 leader_election.cc:288] T 2d7871caf907422d86dfac7cfe4b9561 P 7b0735292a6c448891ef04ecf4df52a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 63e98f6f427648d298613653395a2702 (127.20.113.1:34967), ad71a077f186448fb2fd3100c4ac5c21 (127.20.113.3:44797)
I20260806 09:28:29.006902 23288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:28.998448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.2:48327 (local address 127.20.113.1:34967)
0806 09:28:28.998677 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:28.998684 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:28.998988 (+ 304us) server_negotiation.cc:407] Connection header received
0806 09:28:28.999383 (+ 395us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:28.999390 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:28.999528 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:28.999637 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:28:29.000737 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:29.001871 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:29.004442 (+ 2571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:29.005736 (+ 1294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:29.005949 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:29.006020 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:29.006373 (+ 353us) server_negotiation.cc:299] Negotiation successful
0806 09:28:29.006526 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260806 09:28:29.006928 23294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:28.998421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.2:50857 (local address 127.20.113.3:44797)
0806 09:28:28.998618 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:28.998625 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:28.999437 (+ 812us) server_negotiation.cc:407] Connection header received
0806 09:28:28.999624 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:28.999628 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:28.999689 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:28.999782 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:28:29.000642 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:29.001829 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:29.004579 (+ 2750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:29.005820 (+ 1241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:29.005958 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:29.006020 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:29.006515 (+ 495us) server_negotiation.cc:299] Negotiation successful
0806 09:28:29.006668 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260806 09:28:29.006918 23339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:28.997938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.1:34967 (local address 127.20.113.2:48327)
0806 09:28:28.998866 (+ 928us) negotiation.cc:107] Waiting for socket to connect
0806 09:28:28.998887 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 09:28:28.999127 (+ 240us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:28:28.999814 (+ 687us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:28:28.999819 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:28:28.999850 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:28:29.000511 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:29.000529 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:29.002314 (+ 1785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:29.002319 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:29.004270 (+ 1951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:29.004282 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:29.005981 (+ 1699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:29.005987 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:29.006183 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:29.006221 (+ 38us) client_negotiation.cc:769] Sending connection context
0806 09:28:29.006321 (+ 100us) client_negotiation.cc:240] Negotiation successful
0806 09:28:29.006459 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":779,"mutex_wait_us":76,"spinlock_wait_cycles":2944,"thread_start_us":743,"threads_started":1}
I20260806 09:28:29.007710 23340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:28.998101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.3:44797 (local address 127.20.113.2:50857)
0806 09:28:28.999340 (+ 1239us) negotiation.cc:107] Waiting for socket to connect
0806 09:28:28.999360 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 09:28:28.999492 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:28:28.999818 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:28:28.999822 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:28:28.999885 (+ 63us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:28:29.000464 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:29.000476 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:29.001992 (+ 1516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:29.001999 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:29.004431 (+ 2432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:29.004446 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:29.006171 (+ 1725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:29.006176 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:29.006337 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:29.006368 (+ 31us) client_negotiation.cc:769] Sending connection context
0806 09:28:29.007182 (+ 814us) client_negotiation.cc:240] Negotiation successful
0806 09:28:29.007340 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1114,"mutex_wait_us":8,"spinlock_wait_cycles":4352,"thread_start_us":930,"threads_started":1}
I20260806 09:28:29.007999 22795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d7871caf907422d86dfac7cfe4b9561" candidate_uuid: "7b0735292a6c448891ef04ecf4df52a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63e98f6f427648d298613653395a2702" is_pre_election: true
I20260806 09:28:29.008565 22795 raft_consensus.cc:2454] T 2d7871caf907422d86dfac7cfe4b9561 P 63e98f6f427648d298613653395a2702 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7b0735292a6c448891ef04ecf4df52a2 in term 0.
I20260806 09:28:29.009246 22905 leader_election.cc:302] T 2d7871caf907422d86dfac7cfe4b9561 P 7b0735292a6c448891ef04ecf4df52a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 63e98f6f427648d298613653395a2702, 7b0735292a6c448891ef04ecf4df52a2; no voters:
I20260806 09:28:29.009559 23200 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d7871caf907422d86dfac7cfe4b9561" candidate_uuid: "7b0735292a6c448891ef04ecf4df52a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad71a077f186448fb2fd3100c4ac5c21" is_pre_election: true
I20260806 09:28:29.009733 23303 raft_consensus.cc:2781] T 2d7871caf907422d86dfac7cfe4b9561 P 7b0735292a6c448891ef04ecf4df52a2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 09:28:29.009856 23303 raft_consensus.cc:482] T 2d7871caf907422d86dfac7cfe4b9561 P 7b0735292a6c448891ef04ecf4df52a2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 09:28:29.009871 23200 raft_consensus.cc:2454] T 2d7871caf907422d86dfac7cfe4b9561 P ad71a077f186448fb2fd3100c4ac5c21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7b0735292a6c448891ef04ecf4df52a2 in term 0.
I20260806 09:28:29.009948 23303 raft_consensus.cc:3037] T 2d7871caf907422d86dfac7cfe4b9561 P 7b0735292a6c448891ef04ecf4df52a2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:28:29.013406 23303 raft_consensus.cc:504] T 2d7871caf907422d86dfac7cfe4b9561 P 7b0735292a6c448891ef04ecf4df52a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63e98f6f427648d298613653395a2702" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 34967 } } peers { permanent_uuid: "7b0735292a6c448891ef04ecf4df52a2" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 42551 } } peers { permanent_uuid: "ad71a077f186448fb2fd3100c4ac5c21" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 44797 } }
I20260806 09:28:29.014142 23303 leader_election.cc:288] T 2d7871caf907422d86dfac7cfe4b9561 P 7b0735292a6c448891ef04ecf4df52a2 [CANDIDATE]: Term 1 election: Requested vote from peers 63e98f6f427648d298613653395a2702 (127.20.113.1:34967), ad71a077f186448fb2fd3100c4ac5c21 (127.20.113.3:44797)
I20260806 09:28:29.014665 23200 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d7871caf907422d86dfac7cfe4b9561" candidate_uuid: "7b0735292a6c448891ef04ecf4df52a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad71a077f186448fb2fd3100c4ac5c21"
I20260806 09:28:29.014895 23200 raft_consensus.cc:3037] T 2d7871caf907422d86dfac7cfe4b9561 P ad71a077f186448fb2fd3100c4ac5c21 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:28:29.016572 22795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d7871caf907422d86dfac7cfe4b9561" candidate_uuid: "7b0735292a6c448891ef04ecf4df52a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63e98f6f427648d298613653395a2702"
I20260806 09:28:29.016783 22795 raft_consensus.cc:3037] T 2d7871caf907422d86dfac7cfe4b9561 P 63e98f6f427648d298613653395a2702 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:28:29.018002 23200 raft_consensus.cc:2454] T 2d7871caf907422d86dfac7cfe4b9561 P ad71a077f186448fb2fd3100c4ac5c21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7b0735292a6c448891ef04ecf4df52a2 in term 1.
I20260806 09:28:29.018415 22909 leader_election.cc:302] T 2d7871caf907422d86dfac7cfe4b9561 P 7b0735292a6c448891ef04ecf4df52a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7b0735292a6c448891ef04ecf4df52a2, ad71a077f186448fb2fd3100c4ac5c21; no voters:
I20260806 09:28:29.018725 23303 raft_consensus.cc:2781] T 2d7871caf907422d86dfac7cfe4b9561 P 7b0735292a6c448891ef04ecf4df52a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:28:29.019255 23303 raft_consensus.cc:686] T 2d7871caf907422d86dfac7cfe4b9561 P 7b0735292a6c448891ef04ecf4df52a2 [term 1 LEADER]: Becoming Leader. State: Replica: 7b0735292a6c448891ef04ecf4df52a2, State: Running, Role: LEADER
I20260806 09:28:29.020084 22795 raft_consensus.cc:2454] T 2d7871caf907422d86dfac7cfe4b9561 P 63e98f6f427648d298613653395a2702 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7b0735292a6c448891ef04ecf4df52a2 in term 1.
I20260806 09:28:29.020032 23303 consensus_queue.cc:237] T 2d7871caf907422d86dfac7cfe4b9561 P 7b0735292a6c448891ef04ecf4df52a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63e98f6f427648d298613653395a2702" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 34967 } } peers { permanent_uuid: "7b0735292a6c448891ef04ecf4df52a2" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 42551 } } peers { permanent_uuid: "ad71a077f186448fb2fd3100c4ac5c21" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 44797 } }
I20260806 09:28:29.023813 22583 catalog_manager.cc:5495] T 2d7871caf907422d86dfac7cfe4b9561 P 7b0735292a6c448891ef04ecf4df52a2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7b0735292a6c448891ef04ecf4df52a2 (127.20.113.2). New cstate: current_term: 1 leader_uuid: "7b0735292a6c448891ef04ecf4df52a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63e98f6f427648d298613653395a2702" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 34967 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7b0735292a6c448891ef04ecf4df52a2" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 42551 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ad71a077f186448fb2fd3100c4ac5c21" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 44797 } health_report { overall_health: UNKNOWN } } }
Aug 06 09:28:29 dist-test-slave-45dx krb5kdc[22520](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008506, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.62@KRBTEST.COM for kudu/127.20.113.62@KRBTEST.COM
I20260806 09:28:29.078330 23348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:29.062950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:37651 (local address 127.0.0.1:52952)
0806 09:28:29.063687 (+ 737us) negotiation.cc:107] Waiting for socket to connect
0806 09:28:29.063711 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 09:28:29.063909 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:28:29.064275 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:28:29.064280 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:28:29.065022 (+ 742us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:28:29.065252 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:29.065261 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:29.066512 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:29.066522 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:29.068089 (+ 1567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:29.068106 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:29.068895 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:29.068904 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:29.069090 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:29.069830 (+ 740us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:28:29.069855 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:28:29.072690 (+ 2835us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:28:29.075832 (+ 3142us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:28:29.075845 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:28:29.075853 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:28:29.076249 (+ 396us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:28:29.076657 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:28:29.076661 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:28:29.076665 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:28:29.076806 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:28:29.077310 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:28:29.077317 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:28:29.077490 (+ 173us) client_negotiation.cc:769] Sending connection context
0806 09:28:29.077639 (+ 149us) client_negotiation.cc:240] Negotiation successful
0806 09:28:29.077917 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556,"thread_start_us":376,"threads_started":1}
I20260806 09:28:29.078393 23350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:29.063026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52952 (local address 127.20.113.62:37651)
0806 09:28:29.064012 (+ 986us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:29.064020 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:29.064046 (+ 26us) server_negotiation.cc:407] Connection header received
0806 09:28:29.064085 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:29.064088 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:29.064138 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:29.064246 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:28:29.065419 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:29.066322 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:29.068248 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:29.068722 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:29.068817 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:29.072873 (+ 4056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:28:29.072878 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:28:29.072881 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:28:29.072910 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:28:29.075699 (+ 2789us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:29.076407 (+ 708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:29.076412 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:29.076420 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:29.076544 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:29.076923 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:29.076928 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:29.076930 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:29.077189 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:28:29.077343 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:29.077729 (+ 386us) server_negotiation.cc:299] Negotiation successful
0806 09:28:29.078037 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":127,"server-negotiator.queue_time_us":853,"thread_start_us":679,"threads_started":1}
I20260806 09:28:29.083807 22583 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.20.113.62@KRBTEST.COM'} at 127.0.0.1:52952:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 09:28:29.085001 22583 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 09:28:29.091711 22771 tablet_service.cc:1467] Processing CreateTablet for tablet cb9ce32c19874a53adff6f84bb3655f1 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c3968c23c84c4c46a0c4be97dba7d5c4]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 09:28:29.092341 22771 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb9ce32c19874a53adff6f84bb3655f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:28:29.092157 23165 tablet_service.cc:1467] Processing CreateTablet for tablet cb9ce32c19874a53adff6f84bb3655f1 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c3968c23c84c4c46a0c4be97dba7d5c4]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 09:28:29.092137 22968 tablet_service.cc:1467] Processing CreateTablet for tablet cb9ce32c19874a53adff6f84bb3655f1 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c3968c23c84c4c46a0c4be97dba7d5c4]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 09:28:29.092721 23165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb9ce32c19874a53adff6f84bb3655f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:28:29.092746 22968 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb9ce32c19874a53adff6f84bb3655f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:28:29.099897 23296 tablet_bootstrap.cc:492] T cb9ce32c19874a53adff6f84bb3655f1 P 63e98f6f427648d298613653395a2702: Bootstrap starting.
I20260806 09:28:29.100406 23295 tablet_bootstrap.cc:492] T cb9ce32c19874a53adff6f84bb3655f1 P 7b0735292a6c448891ef04ecf4df52a2: Bootstrap starting.
I20260806 09:28:29.101944 23296 tablet_bootstrap.cc:654] T cb9ce32c19874a53adff6f84bb3655f1 P 63e98f6f427648d298613653395a2702: Neither blocks nor log segments found. Creating new log.
I20260806 09:28:29.102247 23295 tablet_bootstrap.cc:654] T cb9ce32c19874a53adff6f84bb3655f1 P 7b0735292a6c448891ef04ecf4df52a2: Neither blocks nor log segments found. Creating new log.
I20260806 09:28:29.103267 23298 tablet_bootstrap.cc:492] T cb9ce32c19874a53adff6f84bb3655f1 P ad71a077f186448fb2fd3100c4ac5c21: Bootstrap starting.
I20260806 09:28:29.105537 23298 tablet_bootstrap.cc:654] T cb9ce32c19874a53adff6f84bb3655f1 P ad71a077f186448fb2fd3100c4ac5c21: Neither blocks nor log segments found. Creating new log.
I20260806 09:28:29.108156 23298 tablet_bootstrap.cc:492] T cb9ce32c19874a53adff6f84bb3655f1 P ad71a077f186448fb2fd3100c4ac5c21: No bootstrap required, opened a new log
I20260806 09:28:29.108386 23298 ts_tablet_manager.cc:1397] T cb9ce32c19874a53adff6f84bb3655f1 P ad71a077f186448fb2fd3100c4ac5c21: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.004s
I20260806 09:28:29.109750 23298 raft_consensus.cc:348] T cb9ce32c19874a53adff6f84bb3655f1 P ad71a077f186448fb2fd3100c4ac5c21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63e98f6f427648d298613653395a2702" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 34967 } } peers { permanent_uuid: "7b0735292a6c448891ef04ecf4df52a2" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 42551 } } peers { permanent_uuid: "ad71a077f186448fb2fd3100c4ac5c21" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 44797 } }
I20260806 09:28:29.110029 23298 raft_consensus.cc:374] T cb9ce32c19874a53adff6f84bb3655f1 P ad71a077f186448fb2fd3100c4ac5c21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:28:29.110131 23298 raft_consensus.cc:729] T cb9ce32c19874a53adff6f84bb3655f1 P ad71a077f186448fb2fd3100c4ac5c21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad71a077f186448fb2fd3100c4ac5c21, State: Initialized, Role: FOLLOWER
I20260806 09:28:29.110564 23298 consensus_queue.cc:260] T cb9ce32c19874a53adff6f84bb3655f1 P ad71a077f186448fb2fd3100c4ac5c21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63e98f6f427648d298613653395a2702" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 34967 } } peers { permanent_uuid: "7b0735292a6c448891ef04ecf4df52a2" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 42551 } } peers { permanent_uuid: "ad71a077f186448fb2fd3100c4ac5c21" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 44797 } }
I20260806 09:28:29.111363 23298 ts_tablet_manager.cc:1428] T cb9ce32c19874a53adff6f84bb3655f1 P ad71a077f186448fb2fd3100c4ac5c21: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260806 09:28:29.111215 23304 tablet_replica.cc:439] T cb9ce32c19874a53adff6f84bb3655f1 P ad71a077f186448fb2fd3100c4ac5c21: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63e98f6f427648d298613653395a2702" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 34967 } } peers { permanent_uuid: "7b0735292a6c448891ef04ecf4df52a2" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 42551 } } peers { permanent_uuid: "ad71a077f186448fb2fd3100c4ac5c21" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 44797 } } }
I20260806 09:28:29.111581 23304 tablet_replica.cc:442] T cb9ce32c19874a53adff6f84bb3655f1 P ad71a077f186448fb2fd3100c4ac5c21: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 09:28:29.113612 23295 tablet_bootstrap.cc:492] T cb9ce32c19874a53adff6f84bb3655f1 P 7b0735292a6c448891ef04ecf4df52a2: No bootstrap required, opened a new log
I20260806 09:28:29.113816 23295 ts_tablet_manager.cc:1397] T cb9ce32c19874a53adff6f84bb3655f1 P 7b0735292a6c448891ef04ecf4df52a2: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.006s
I20260806 09:28:29.114670 23296 tablet_bootstrap.cc:492] T cb9ce32c19874a53adff6f84bb3655f1 P 63e98f6f427648d298613653395a2702: No bootstrap required, opened a new log
I20260806 09:28:29.114650 23295 raft_consensus.cc:348] T cb9ce32c19874a53adff6f84bb3655f1 P 7b0735292a6c448891ef04ecf4df52a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63e98f6f427648d298613653395a2702" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 34967 } } peers { permanent_uuid: "7b0735292a6c448891ef04ecf4df52a2" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 42551 } } peers { permanent_uuid: "ad71a077f186448fb2fd3100c4ac5c21" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 44797 } }
I20260806 09:28:29.114924 23295 raft_consensus.cc:374] T cb9ce32c19874a53adff6f84bb3655f1 P 7b0735292a6c448891ef04ecf4df52a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:28:29.115039 23295 raft_consensus.cc:729] T cb9ce32c19874a53adff6f84bb3655f1 P 7b0735292a6c448891ef04ecf4df52a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b0735292a6c448891ef04ecf4df52a2, State: Initialized, Role: FOLLOWER
I20260806 09:28:29.114977 23296 ts_tablet_manager.cc:1397] T cb9ce32c19874a53adff6f84bb3655f1 P 63e98f6f427648d298613653395a2702: Time spent bootstrapping tablet: real 0.015s user 0.008s sys 0.003s
I20260806 09:28:29.115321 23295 consensus_queue.cc:260] T cb9ce32c19874a53adff6f84bb3655f1 P 7b0735292a6c448891ef04ecf4df52a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63e98f6f427648d298613653395a2702" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 34967 } } peers { permanent_uuid: "7b0735292a6c448891ef04ecf4df52a2" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 42551 } } peers { permanent_uuid: "ad71a077f186448fb2fd3100c4ac5c21" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 44797 } }
I20260806 09:28:29.115967 23295 ts_tablet_manager.cc:1428] T cb9ce32c19874a53adff6f84bb3655f1 P 7b0735292a6c448891ef04ecf4df52a2: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260806 09:28:29.115783 23303 tablet_replica.cc:439] T cb9ce32c19874a53adff6f84bb3655f1 P 7b0735292a6c448891ef04ecf4df52a2: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63e98f6f427648d298613653395a2702" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 34967 } } peers { permanent_uuid: "7b0735292a6c448891ef04ecf4df52a2" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 42551 } } peers { permanent_uuid: "ad71a077f186448fb2fd3100c4ac5c21" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 44797 } } }
I20260806 09:28:29.116202 23303 tablet_replica.cc:442] T cb9ce32c19874a53adff6f84bb3655f1 P 7b0735292a6c448891ef04ecf4df52a2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 09:28:29.117105 23296 raft_consensus.cc:348] T cb9ce32c19874a53adff6f84bb3655f1 P 63e98f6f427648d298613653395a2702 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63e98f6f427648d298613653395a2702" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 34967 } } peers { permanent_uuid: "7b0735292a6c448891ef04ecf4df52a2" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 42551 } } peers { permanent_uuid: "ad71a077f186448fb2fd3100c4ac5c21" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 44797 } }
I20260806 09:28:29.117379 23296 raft_consensus.cc:374] T cb9ce32c19874a53adff6f84bb3655f1 P 63e98f6f427648d298613653395a2702 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:28:29.117522 23296 raft_consensus.cc:729] T cb9ce32c19874a53adff6f84bb3655f1 P 63e98f6f427648d298613653395a2702 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63e98f6f427648d298613653395a2702, State: Initialized, Role: FOLLOWER
I20260806 09:28:29.117900 23296 consensus_queue.cc:260] T cb9ce32c19874a53adff6f84bb3655f1 P 63e98f6f427648d298613653395a2702 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63e98f6f427648d298613653395a2702" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 34967 } } peers { permanent_uuid: "7b0735292a6c448891ef04ecf4df52a2" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 42551 } } peers { permanent_uuid: "ad71a077f186448fb2fd3100c4ac5c21" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 44797 } }
I20260806 09:28:29.118453 23323 tablet_replica.cc:439] T cb9ce32c19874a53adff6f84bb3655f1 P 63e98f6f427648d298613653395a2702: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63e98f6f427648d298613653395a2702" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 34967 } } peers { permanent_uuid: "7b0735292a6c448891ef04ecf4df52a2" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 42551 } } peers { permanent_uuid: "ad71a077f186448fb2fd3100c4ac5c21" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 44797 } } }
I20260806 09:28:29.118880 23323 tablet_replica.cc:442] T cb9ce32c19874a53adff6f84bb3655f1 P 63e98f6f427648d298613653395a2702: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 09:28:29.119391 23296 ts_tablet_manager.cc:1428] T cb9ce32c19874a53adff6f84bb3655f1 P 63e98f6f427648d298613653395a2702: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260806 09:28:29.143982 23323 raft_consensus.cc:482] T cb9ce32c19874a53adff6f84bb3655f1 P 63e98f6f427648d298613653395a2702 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 09:28:29.144184 23323 raft_consensus.cc:504] T cb9ce32c19874a53adff6f84bb3655f1 P 63e98f6f427648d298613653395a2702 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63e98f6f427648d298613653395a2702" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 34967 } } peers { permanent_uuid: "7b0735292a6c448891ef04ecf4df52a2" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 42551 } } peers { permanent_uuid: "ad71a077f186448fb2fd3100c4ac5c21" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 44797 } }
I20260806 09:28:29.145027 23323 leader_election.cc:288] T cb9ce32c19874a53adff6f84bb3655f1 P 63e98f6f427648d298613653395a2702 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7b0735292a6c448891ef04ecf4df52a2 (127.20.113.2:42551), ad71a077f186448fb2fd3100c4ac5c21 (127.20.113.3:44797)
I20260806 09:28:29.145594 23005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb9ce32c19874a53adff6f84bb3655f1" candidate_uuid: "63e98f6f427648d298613653395a2702" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b0735292a6c448891ef04ecf4df52a2" is_pre_election: true
I20260806 09:28:29.145641 23200 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb9ce32c19874a53adff6f84bb3655f1" candidate_uuid: "63e98f6f427648d298613653395a2702" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad71a077f186448fb2fd3100c4ac5c21" is_pre_election: true
I20260806 09:28:29.145855 23005 raft_consensus.cc:2454] T cb9ce32c19874a53adff6f84bb3655f1 P 7b0735292a6c448891ef04ecf4df52a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 63e98f6f427648d298613653395a2702 in term 0.
I20260806 09:28:29.146005 23200 raft_consensus.cc:2454] T cb9ce32c19874a53adff6f84bb3655f1 P ad71a077f186448fb2fd3100c4ac5c21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 63e98f6f427648d298613653395a2702 in term 0.
I20260806 09:28:29.146374 22705 leader_election.cc:302] T cb9ce32c19874a53adff6f84bb3655f1 P 63e98f6f427648d298613653395a2702 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 63e98f6f427648d298613653395a2702, 7b0735292a6c448891ef04ecf4df52a2; no voters:
I20260806 09:28:29.146807 23323 raft_consensus.cc:2781] T cb9ce32c19874a53adff6f84bb3655f1 P 63e98f6f427648d298613653395a2702 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 09:28:29.146941 23323 raft_consensus.cc:482] T cb9ce32c19874a53adff6f84bb3655f1 P 63e98f6f427648d298613653395a2702 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 09:28:29.147030 23323 raft_consensus.cc:3037] T cb9ce32c19874a53adff6f84bb3655f1 P 63e98f6f427648d298613653395a2702 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:28:29.149550 23323 raft_consensus.cc:504] T cb9ce32c19874a53adff6f84bb3655f1 P 63e98f6f427648d298613653395a2702 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63e98f6f427648d298613653395a2702" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 34967 } } peers { permanent_uuid: "7b0735292a6c448891ef04ecf4df52a2" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 42551 } } peers { permanent_uuid: "ad71a077f186448fb2fd3100c4ac5c21" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 44797 } }
I20260806 09:28:29.150128 23323 leader_election.cc:288] T cb9ce32c19874a53adff6f84bb3655f1 P 63e98f6f427648d298613653395a2702 [CANDIDATE]: Term 1 election: Requested vote from peers 7b0735292a6c448891ef04ecf4df52a2 (127.20.113.2:42551), ad71a077f186448fb2fd3100c4ac5c21 (127.20.113.3:44797)
I20260806 09:28:29.150624 23005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb9ce32c19874a53adff6f84bb3655f1" candidate_uuid: "63e98f6f427648d298613653395a2702" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b0735292a6c448891ef04ecf4df52a2"
I20260806 09:28:29.150707 23200 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb9ce32c19874a53adff6f84bb3655f1" candidate_uuid: "63e98f6f427648d298613653395a2702" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad71a077f186448fb2fd3100c4ac5c21"
I20260806 09:28:29.150880 23005 raft_consensus.cc:3037] T cb9ce32c19874a53adff6f84bb3655f1 P 7b0735292a6c448891ef04ecf4df52a2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:28:29.150982 23200 raft_consensus.cc:3037] T cb9ce32c19874a53adff6f84bb3655f1 P ad71a077f186448fb2fd3100c4ac5c21 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:28:29.153200 23005 raft_consensus.cc:2454] T cb9ce32c19874a53adff6f84bb3655f1 P 7b0735292a6c448891ef04ecf4df52a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 63e98f6f427648d298613653395a2702 in term 1.
I20260806 09:28:29.153627 22705 leader_election.cc:302] T cb9ce32c19874a53adff6f84bb3655f1 P 63e98f6f427648d298613653395a2702 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 63e98f6f427648d298613653395a2702, 7b0735292a6c448891ef04ecf4df52a2; no voters:
I20260806 09:28:29.153888 23323 raft_consensus.cc:2781] T cb9ce32c19874a53adff6f84bb3655f1 P 63e98f6f427648d298613653395a2702 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:28:29.154000 23200 raft_consensus.cc:2454] T cb9ce32c19874a53adff6f84bb3655f1 P ad71a077f186448fb2fd3100c4ac5c21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 63e98f6f427648d298613653395a2702 in term 1.
I20260806 09:28:29.154040 23323 raft_consensus.cc:686] T cb9ce32c19874a53adff6f84bb3655f1 P 63e98f6f427648d298613653395a2702 [term 1 LEADER]: Becoming Leader. State: Replica: 63e98f6f427648d298613653395a2702, State: Running, Role: LEADER
I20260806 09:28:29.154295 23323 consensus_queue.cc:237] T cb9ce32c19874a53adff6f84bb3655f1 P 63e98f6f427648d298613653395a2702 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63e98f6f427648d298613653395a2702" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 34967 } } peers { permanent_uuid: "7b0735292a6c448891ef04ecf4df52a2" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 42551 } } peers { permanent_uuid: "ad71a077f186448fb2fd3100c4ac5c21" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 44797 } }
I20260806 09:28:29.156206 23302 tablet_replica.cc:439] T cb9ce32c19874a53adff6f84bb3655f1 P 63e98f6f427648d298613653395a2702: TxnStatusTablet state changed. Reason: New leader 63e98f6f427648d298613653395a2702. Latest consensus state: current_term: 1 leader_uuid: "63e98f6f427648d298613653395a2702" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63e98f6f427648d298613653395a2702" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 34967 } } peers { permanent_uuid: "7b0735292a6c448891ef04ecf4df52a2" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 42551 } } peers { permanent_uuid: "ad71a077f186448fb2fd3100c4ac5c21" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 44797 } } }
I20260806 09:28:29.156546 23302 tablet_replica.cc:442] T cb9ce32c19874a53adff6f84bb3655f1 P 63e98f6f427648d298613653395a2702: This TxnStatusTablet replica's current role is: LEADER
I20260806 09:28:29.157563 23356 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 09:28:29.157855 23356 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 09:28:29.158286 22581 catalog_manager.cc:5495] T cb9ce32c19874a53adff6f84bb3655f1 P 63e98f6f427648d298613653395a2702 reported cstate change: term changed from 0 to 1, leader changed from <none> to 63e98f6f427648d298613653395a2702 (127.20.113.1). New cstate: current_term: 1 leader_uuid: "63e98f6f427648d298613653395a2702" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63e98f6f427648d298613653395a2702" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 34967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7b0735292a6c448891ef04ecf4df52a2" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 42551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ad71a077f186448fb2fd3100c4ac5c21" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 44797 } health_report { overall_health: UNKNOWN } } }
I20260806 09:28:29.189867 23348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:29.184344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.1:34967 (local address 127.0.0.1:58702)
0806 09:28:29.184574 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0806 09:28:29.184597 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 09:28:29.184701 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:28:29.185095 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:28:29.185098 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:28:29.185107 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 09:28:29.185256 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:29.185263 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:29.186850 (+ 1587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:29.186853 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:29.187740 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:29.187751 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:29.188744 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:29.188748 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:29.188925 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:29.188978 (+ 53us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 09:28:29.189432 (+ 454us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 09:28:29.189447 (+ 15us) client_negotiation.cc:769] Sending connection context
0806 09:28:29.189510 (+ 63us) client_negotiation.cc:240] Negotiation successful
0806 09:28:29.189617 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260806 09:28:29.190001 23288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:29.184497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58702 (local address 127.20.113.1:34967)
0806 09:28:29.184721 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:29.184728 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:29.184750 (+ 22us) server_negotiation.cc:407] Connection header received
0806 09:28:29.184824 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:29.184829 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:29.184961 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:29.185070 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 09:28:29.185402 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:29.186716 (+ 1314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:29.187881 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:29.188595 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:29.188733 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:29.189150 (+ 417us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 09:28:29.189307 (+ 157us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 09:28:29.189438 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:29.189566 (+ 128us) server_negotiation.cc:299] Negotiation successful
0806 09:28:29.189646 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 09:28:29.194214 23200 raft_consensus.cc:1261] T cb9ce32c19874a53adff6f84bb3655f1 P ad71a077f186448fb2fd3100c4ac5c21 [term 1 FOLLOWER]: Refusing update from remote peer 63e98f6f427648d298613653395a2702: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 09:28:29.194293 23005 raft_consensus.cc:1261] T cb9ce32c19874a53adff6f84bb3655f1 P 7b0735292a6c448891ef04ecf4df52a2 [term 1 FOLLOWER]: Refusing update from remote peer 63e98f6f427648d298613653395a2702: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 09:28:29.194813 23323 consensus_queue.cc:1035] T cb9ce32c19874a53adff6f84bb3655f1 P 63e98f6f427648d298613653395a2702 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad71a077f186448fb2fd3100c4ac5c21" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 44797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 09:28:29.195196 23302 consensus_queue.cc:1035] T cb9ce32c19874a53adff6f84bb3655f1 P 63e98f6f427648d298613653395a2702 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7b0735292a6c448891ef04ecf4df52a2" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 42551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 09:28:29.201424 23303 tablet_replica.cc:439] T cb9ce32c19874a53adff6f84bb3655f1 P 7b0735292a6c448891ef04ecf4df52a2: TxnStatusTablet state changed. Reason: New leader 63e98f6f427648d298613653395a2702. Latest consensus state: current_term: 1 leader_uuid: "63e98f6f427648d298613653395a2702" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63e98f6f427648d298613653395a2702" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 34967 } } peers { permanent_uuid: "7b0735292a6c448891ef04ecf4df52a2" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 42551 } } peers { permanent_uuid: "ad71a077f186448fb2fd3100c4ac5c21" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 44797 } } }
I20260806 09:28:29.201867 23303 tablet_replica.cc:442] T cb9ce32c19874a53adff6f84bb3655f1 P 7b0735292a6c448891ef04ecf4df52a2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 09:28:29.202754 23304 tablet_replica.cc:439] T cb9ce32c19874a53adff6f84bb3655f1 P ad71a077f186448fb2fd3100c4ac5c21: TxnStatusTablet state changed. Reason: New leader 63e98f6f427648d298613653395a2702. Latest consensus state: current_term: 1 leader_uuid: "63e98f6f427648d298613653395a2702" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63e98f6f427648d298613653395a2702" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 34967 } } peers { permanent_uuid: "7b0735292a6c448891ef04ecf4df52a2" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 42551 } } peers { permanent_uuid: "ad71a077f186448fb2fd3100c4ac5c21" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 44797 } } }
I20260806 09:28:29.203063 23304 tablet_replica.cc:442] T cb9ce32c19874a53adff6f84bb3655f1 P ad71a077f186448fb2fd3100c4ac5c21: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 09:28:29.205005 23323 tablet_replica.cc:439] T cb9ce32c19874a53adff6f84bb3655f1 P 63e98f6f427648d298613653395a2702: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "63e98f6f427648d298613653395a2702" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63e98f6f427648d298613653395a2702" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 34967 } } peers { permanent_uuid: "7b0735292a6c448891ef04ecf4df52a2" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 42551 } } peers { permanent_uuid: "ad71a077f186448fb2fd3100c4ac5c21" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 44797 } } }
I20260806 09:28:29.205142 23316 mvcc.cc:205] Tried to move back new op lower bound from 7315490853653176320 to 7315490853498511360. Current Snapshot: MvccSnapshot[applied={T|T < 7315490853653176320}]
I20260806 09:28:29.205263 23323 tablet_replica.cc:442] T cb9ce32c19874a53adff6f84bb3655f1 P 63e98f6f427648d298613653395a2702: This TxnStatusTablet replica's current role is: LEADER
I20260806 09:28:29.205706 23312 tablet_replica.cc:439] T cb9ce32c19874a53adff6f84bb3655f1 P 63e98f6f427648d298613653395a2702: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "63e98f6f427648d298613653395a2702" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63e98f6f427648d298613653395a2702" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 34967 } } peers { permanent_uuid: "7b0735292a6c448891ef04ecf4df52a2" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 42551 } } peers { permanent_uuid: "ad71a077f186448fb2fd3100c4ac5c21" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 44797 } } }
I20260806 09:28:29.206106 23312 tablet_replica.cc:442] T cb9ce32c19874a53adff6f84bb3655f1 P 63e98f6f427648d298613653395a2702: This TxnStatusTablet replica's current role is: LEADER
I20260806 09:28:29.208170 23303 tablet_replica.cc:439] T cb9ce32c19874a53adff6f84bb3655f1 P 7b0735292a6c448891ef04ecf4df52a2: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "63e98f6f427648d298613653395a2702" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63e98f6f427648d298613653395a2702" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 34967 } } peers { permanent_uuid: "7b0735292a6c448891ef04ecf4df52a2" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 42551 } } peers { permanent_uuid: "ad71a077f186448fb2fd3100c4ac5c21" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 44797 } } }
I20260806 09:28:29.208491 23303 tablet_replica.cc:442] T cb9ce32c19874a53adff6f84bb3655f1 P 7b0735292a6c448891ef04ecf4df52a2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 09:28:29.209609 23304 tablet_replica.cc:439] T cb9ce32c19874a53adff6f84bb3655f1 P ad71a077f186448fb2fd3100c4ac5c21: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "63e98f6f427648d298613653395a2702" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63e98f6f427648d298613653395a2702" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 34967 } } peers { permanent_uuid: "7b0735292a6c448891ef04ecf4df52a2" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 42551 } } peers { permanent_uuid: "ad71a077f186448fb2fd3100c4ac5c21" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 44797 } } }
I20260806 09:28:29.209894 23304 tablet_replica.cc:442] T cb9ce32c19874a53adff6f84bb3655f1 P ad71a077f186448fb2fd3100c4ac5c21: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 09:28:29.221102 23291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:29.215076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32948 (local address 127.20.113.2:42551)
0806 09:28:29.215272 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:29.215280 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:29.215731 (+ 451us) server_negotiation.cc:407] Connection header received
0806 09:28:29.215876 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:29.215881 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:29.216004 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:29.216155 (+ 151us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 09:28:29.216566 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:29.217434 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:29.218685 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:29.219132 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:29.219305 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:29.219746 (+ 441us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 09:28:29.220385 (+ 639us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 09:28:29.220532 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:29.220729 (+ 197us) server_negotiation.cc:299] Negotiation successful
0806 09:28:29.220810 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 09:28:29.239269 23362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:29.232518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.1:34967 (local address 127.20.113.2:48731)
0806 09:28:29.233135 (+ 617us) negotiation.cc:107] Waiting for socket to connect
0806 09:28:29.233150 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 09:28:29.233265 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:28:29.233660 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:28:29.233664 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:28:29.233681 (+ 17us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 09:28:29.233970 (+ 289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:29.233978 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:29.235588 (+ 1610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:29.235591 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:29.236783 (+ 1192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:29.236796 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:29.237965 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:29.237974 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:29.238184 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:29.238243 (+ 59us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 09:28:29.238706 (+ 463us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 09:28:29.238715 (+ 9us) client_negotiation.cc:769] Sending connection context
0806 09:28:29.238779 (+ 64us) client_negotiation.cc:240] Negotiation successful
0806 09:28:29.238909 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":317,"threads_started":1}
I20260806 09:28:29.239269 23288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:29.232650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.2:48731 (local address 127.20.113.1:34967)
0806 09:28:29.232891 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:29.232898 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:29.233240 (+ 342us) server_negotiation.cc:407] Connection header received
0806 09:28:29.233410 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:29.233417 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:29.233515 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:29.233622 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 09:28:29.234123 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:29.235447 (+ 1324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:29.236933 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:29.237757 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:29.237932 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:29.238426 (+ 494us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 09:28:29.238595 (+ 169us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 09:28:29.238707 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:29.238832 (+ 125us) server_negotiation.cc:299] Negotiation successful
0806 09:28:29.238946 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 09:28:29.251348 23200 raft_consensus.cc:1261] T 2d7871caf907422d86dfac7cfe4b9561 P ad71a077f186448fb2fd3100c4ac5c21 [term 1 FOLLOWER]: Refusing update from remote peer 7b0735292a6c448891ef04ecf4df52a2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 09:28:29.251711 22795 raft_consensus.cc:1261] T 2d7871caf907422d86dfac7cfe4b9561 P 63e98f6f427648d298613653395a2702 [term 1 FOLLOWER]: Refusing update from remote peer 7b0735292a6c448891ef04ecf4df52a2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 09:28:29.252092 23303 consensus_queue.cc:1035] T 2d7871caf907422d86dfac7cfe4b9561 P 7b0735292a6c448891ef04ecf4df52a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad71a077f186448fb2fd3100c4ac5c21" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 44797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 09:28:29.252494 23341 consensus_queue.cc:1035] T 2d7871caf907422d86dfac7cfe4b9561 P 7b0735292a6c448891ef04ecf4df52a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "63e98f6f427648d298613653395a2702" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 34967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 09:28:29.291718 23291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:29.284668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.1:34027 (local address 127.20.113.2:42551)
0806 09:28:29.285010 (+ 342us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:29.285017 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:29.285438 (+ 421us) server_negotiation.cc:407] Connection header received
0806 09:28:29.285689 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:29.285693 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:29.285758 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:29.285833 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 09:28:29.286429 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:29.287464 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:29.289808 (+ 2344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:29.290245 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:29.290352 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:29.290962 (+ 610us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 09:28:29.291102 (+ 140us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 09:28:29.291194 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:29.291335 (+ 141us) server_negotiation.cc:299] Negotiation successful
0806 09:28:29.291416 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187}
I20260806 09:28:29.291910 23370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:29.284477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.2:42551 (local address 127.20.113.1:34027)
0806 09:28:29.285324 (+ 847us) negotiation.cc:107] Waiting for socket to connect
0806 09:28:29.285350 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 09:28:29.285532 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:28:29.285875 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:28:29.285880 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:28:29.285894 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 09:28:29.286272 (+ 378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:29.286285 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:29.287625 (+ 1340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:29.287630 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:29.289665 (+ 2035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:29.289682 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:29.290452 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:29.290465 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:29.290666 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:29.290762 (+ 96us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 09:28:29.291196 (+ 434us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 09:28:29.291212 (+ 16us) client_negotiation.cc:769] Sending connection context
0806 09:28:29.291310 (+ 98us) client_negotiation.cc:240] Negotiation successful
0806 09:28:29.291452 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":617,"thread_start_us":433,"threads_started":1}
I20260806 09:28:29.337694 22583 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52950:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 09:28:29.338002 22583 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52950:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 09:28:29.341938 22583 catalog_manager.cc:5782] T 00000000000000000000000000000000 P aefb46d3774747bca7b9baf9042e72a3: Sending DeleteTablet for 3 replicas of tablet 2d7871caf907422d86dfac7cfe4b9561
I20260806 09:28:29.343231 22771 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d7871caf907422d86dfac7cfe4b9561 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:28:29 UTC) from {username='kudu', principal='kudu/127.20.113.62@KRBTEST.COM'} at 127.0.0.1:58688
I20260806 09:28:29.343562 23332 tablet_replica.cc:330] T 2d7871caf907422d86dfac7cfe4b9561 P 63e98f6f427648d298613653395a2702: stopping tablet replica
I20260806 09:28:29.343824 23332 raft_consensus.cc:2229] T 2d7871caf907422d86dfac7cfe4b9561 P 63e98f6f427648d298613653395a2702 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 09:28:29.344056 23332 raft_consensus.cc:2258] T 2d7871caf907422d86dfac7cfe4b9561 P 63e98f6f427648d298613653395a2702 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 09:28:29.344786 22968 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d7871caf907422d86dfac7cfe4b9561 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:28:29 UTC) from {username='kudu', principal='kudu/127.20.113.62@KRBTEST.COM'} at 127.0.0.1:32936
I20260806 09:28:29.345081 23334 tablet_replica.cc:330] T 2d7871caf907422d86dfac7cfe4b9561 P 7b0735292a6c448891ef04ecf4df52a2: stopping tablet replica
I20260806 09:28:29.345292 23334 raft_consensus.cc:2229] T 2d7871caf907422d86dfac7cfe4b9561 P 7b0735292a6c448891ef04ecf4df52a2 [term 1 LEADER]: Raft consensus shutting down.
I20260806 09:28:29.345659 23332 ts_tablet_manager.cc:1905] T 2d7871caf907422d86dfac7cfe4b9561 P 63e98f6f427648d298613653395a2702: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 09:28:29.345927 23334 raft_consensus.cc:2258] T 2d7871caf907422d86dfac7cfe4b9561 P 7b0735292a6c448891ef04ecf4df52a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 09:28:29.346948 23165 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d7871caf907422d86dfac7cfe4b9561 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:28:29 UTC) from {username='kudu', principal='kudu/127.20.113.62@KRBTEST.COM'} at 127.0.0.1:37104
I20260806 09:28:29.347223 23333 tablet_replica.cc:330] T 2d7871caf907422d86dfac7cfe4b9561 P ad71a077f186448fb2fd3100c4ac5c21: stopping tablet replica
I20260806 09:28:29.347432 23333 raft_consensus.cc:2229] T 2d7871caf907422d86dfac7cfe4b9561 P ad71a077f186448fb2fd3100c4ac5c21 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 09:28:29.347646 23333 raft_consensus.cc:2258] T 2d7871caf907422d86dfac7cfe4b9561 P ad71a077f186448fb2fd3100c4ac5c21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 09:28:29.349179 23333 ts_tablet_manager.cc:1905] T 2d7871caf907422d86dfac7cfe4b9561 P ad71a077f186448fb2fd3100c4ac5c21: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 09:28:29.354199 23334 ts_tablet_manager.cc:1905] T 2d7871caf907422d86dfac7cfe4b9561 P 7b0735292a6c448891ef04ecf4df52a2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 09:28:29.361706 23332 ts_tablet_manager.cc:1918] T 2d7871caf907422d86dfac7cfe4b9561 P 63e98f6f427648d298613653395a2702: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 09:28:29.362181 23332 log.cc:1196] T 2d7871caf907422d86dfac7cfe4b9561 P 63e98f6f427648d298613653395a2702: Deleting WAL directory at /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786008495763054-20932-0/minicluster-data/ts-0/wal/wals/2d7871caf907422d86dfac7cfe4b9561
I20260806 09:28:29.362972 23332 ts_tablet_manager.cc:1939] T 2d7871caf907422d86dfac7cfe4b9561 P 63e98f6f427648d298613653395a2702: Deleting consensus metadata
I20260806 09:28:29.364370 22562 catalog_manager.cc:4841] TS 63e98f6f427648d298613653395a2702 (127.20.113.1:34967): tablet 2d7871caf907422d86dfac7cfe4b9561 (table test-table [id=6a2e08afffbc4a74a10a0758bb70a64b]) successfully deleted
I20260806 09:28:29.368554 23333 ts_tablet_manager.cc:1918] T 2d7871caf907422d86dfac7cfe4b9561 P ad71a077f186448fb2fd3100c4ac5c21: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 09:28:29.368815 23333 log.cc:1196] T 2d7871caf907422d86dfac7cfe4b9561 P ad71a077f186448fb2fd3100c4ac5c21: Deleting WAL directory at /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786008495763054-20932-0/minicluster-data/ts-2/wal/wals/2d7871caf907422d86dfac7cfe4b9561
I20260806 09:28:29.369395 23333 ts_tablet_manager.cc:1939] T 2d7871caf907422d86dfac7cfe4b9561 P ad71a077f186448fb2fd3100c4ac5c21: Deleting consensus metadata
I20260806 09:28:29.370714 22564 catalog_manager.cc:4841] TS ad71a077f186448fb2fd3100c4ac5c21 (127.20.113.3:44797): tablet 2d7871caf907422d86dfac7cfe4b9561 (table test-table [id=6a2e08afffbc4a74a10a0758bb70a64b]) successfully deleted
I20260806 09:28:29.373847 23334 ts_tablet_manager.cc:1918] T 2d7871caf907422d86dfac7cfe4b9561 P 7b0735292a6c448891ef04ecf4df52a2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 09:28:29.374049 23334 log.cc:1196] T 2d7871caf907422d86dfac7cfe4b9561 P 7b0735292a6c448891ef04ecf4df52a2: Deleting WAL directory at /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786008495763054-20932-0/minicluster-data/ts-1/wal/wals/2d7871caf907422d86dfac7cfe4b9561
I20260806 09:28:29.374644 23334 ts_tablet_manager.cc:1939] T 2d7871caf907422d86dfac7cfe4b9561 P 7b0735292a6c448891ef04ecf4df52a2: Deleting consensus metadata
I20260806 09:28:29.375825 22560 catalog_manager.cc:4841] TS 7b0735292a6c448891ef04ecf4df52a2 (127.20.113.2:42551): tablet 2d7871caf907422d86dfac7cfe4b9561 (table test-table [id=6a2e08afffbc4a74a10a0758bb70a64b]) successfully deleted
Aug 06 09:28:29 dist-test-slave-45dx krb5kdc[22520](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008508, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.20.113.1@KRBTEST.COM
I20260806 09:28:29.398196 23288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:29.380782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58706 (local address 127.20.113.1:34967)
0806 09:28:29.380995 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:29.381004 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:29.381733 (+ 729us) server_negotiation.cc:407] Connection header received
0806 09:28:29.381976 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:29.381986 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:29.382053 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:29.382189 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:28:29.383835 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:29.384719 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:29.386098 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:29.386520 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:29.386668 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:29.390876 (+ 4208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:28:29.390888 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:28:29.390903 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:28:29.390936 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:28:29.394099 (+ 3163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:29.394828 (+ 729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:29.394838 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:29.394841 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:29.394989 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:29.395419 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:29.395429 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:29.395431 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:29.395942 (+ 511us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:28:29.396422 (+ 480us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:29.397513 (+ 1091us) server_negotiation.cc:299] Negotiation successful
0806 09:28:29.397758 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
W20260806 09:28:29.399006 22867 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58706
I20260806 09:28:29.423840 23350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:29.402496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52964 (local address 127.20.113.62:37651)
0806 09:28:29.402764 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:29.402773 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:29.406121 (+ 3348us) server_negotiation.cc:407] Connection header received
0806 09:28:29.406315 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:29.406321 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:29.406408 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:29.406525 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:28:29.408469 (+ 1944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:29.409665 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:29.411592 (+ 1927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:29.412369 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:29.412471 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:29.416208 (+ 3737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:28:29.416216 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:28:29.416221 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:28:29.416266 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:28:29.420005 (+ 3739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:29.421213 (+ 1208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:29.421220 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:29.421223 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:29.421387 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:29.421946 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:29.421951 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:29.421955 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:29.422367 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:28:29.422614 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:29.423216 (+ 602us) server_negotiation.cc:299] Negotiation successful
0806 09:28:29.423464 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
W20260806 09:28:29.425652 22583 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52964
I20260806 09:28:29.428277 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 22684
I20260806 09:28:29.455284 23288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:29.430189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58712 (local address 127.20.113.1:34967)
0806 09:28:29.430384 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:29.430390 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:29.437634 (+ 7244us) server_negotiation.cc:407] Connection header received
0806 09:28:29.437942 (+ 308us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:29.437949 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:29.438060 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:29.438151 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:28:29.440010 (+ 1859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:29.441362 (+ 1352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:29.443465 (+ 2103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:29.444358 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:29.444465 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:29.447809 (+ 3344us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:28:29.447815 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:28:29.447820 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:28:29.447861 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:28:29.451725 (+ 3864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:29.452621 (+ 896us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:29.452627 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:29.452631 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:29.452826 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:29.453337 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:29.453342 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:29.453346 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:29.453689 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:28:29.453977 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:29.454569 (+ 592us) server_negotiation.cc:299] Negotiation successful
0806 09:28:29.454841 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
W20260806 09:28:29.457213 22867 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58712
W20260806 09:28:29.457978 20932 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 22684: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 09:28:29.458257 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 22684
I20260806 09:28:29.494660 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 22880
Aug 06 09:28:29 dist-test-slave-45dx krb5kdc[22520](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008508, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.20.113.2@KRBTEST.COM
I20260806 09:28:29.511423 23291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:29.495625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32954 (local address 127.20.113.2:42551)
0806 09:28:29.495873 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:29.495879 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:29.495900 (+ 21us) server_negotiation.cc:407] Connection header received
0806 09:28:29.495966 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:29.495976 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:29.496047 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:29.496152 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:28:29.497428 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:29.498310 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:29.499584 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:29.500071 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:29.500219 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:29.504233 (+ 4014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:28:29.504246 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:28:29.504261 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:28:29.504351 (+ 90us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:28:29.508528 (+ 4177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:29.509233 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:29.509248 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:29.509252 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:29.509442 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:29.509782 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:29.509788 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:29.509793 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:29.510245 (+ 452us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:28:29.510660 (+ 415us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:29.510810 (+ 150us) server_negotiation.cc:299] Negotiation successful
0806 09:28:29.511035 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
W20260806 09:28:29.512238 23062 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:32954
W20260806 09:28:29.512921 20932 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 22880: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 09:28:29.513093 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 22880
I20260806 09:28:29.544459 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 23076
Aug 06 09:28:29 dist-test-slave-45dx krb5kdc[22520](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008508, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.20.113.3@KRBTEST.COM
I20260806 09:28:29.559892 23386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:29.545239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37110 (local address 127.20.113.3:44797)
0806 09:28:29.545799 (+ 560us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:29.545803 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:29.545820 (+ 17us) server_negotiation.cc:407] Connection header received
0806 09:28:29.545905 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:29.545909 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:29.545972 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:29.546064 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:28:29.547373 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:29.548093 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:29.549327 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:29.549779 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:29.549884 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:29.554007 (+ 4123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:28:29.554024 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:28:29.554041 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:28:29.554079 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:28:29.557125 (+ 3046us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:29.557795 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:29.557802 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:29.557804 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:29.557947 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:29.558319 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:29.558324 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:29.558327 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:29.558683 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:28:29.559009 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:29.559317 (+ 308us) server_negotiation.cc:299] Negotiation successful
0806 09:28:29.559550 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":459,"thread_start_us":324,"threads_started":1}
W20260806 09:28:29.560603 23261 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37110
W20260806 09:28:29.561278 20932 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 23076: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 09:28:29.561441 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 23076
I20260806 09:28:29.589092 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 22540
I20260806 09:28:29.590232 22638 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:28:29.711751 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 22540
2026-08-06T09:28:29Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (4963 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786008495763054-20932-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 09:28:29 dist-test-slave-45dx krb5kdc[23391](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 09:28:29 dist-test-slave-45dx krb5kdc[23391](info): set up 2 sockets
Aug 06 09:28:29 dist-test-slave-45dx krb5kdc[23391](info): commencing operation
krb5kdc: starting...
W20260806 09:28:31.006610 20932 mini_kdc.cc:121] Time spent starting KDC: real 1.246s user 0.000s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786008495763054-20932-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786008495763054-20932-0/krb5kdc/test-user.keytab.
Aug 06 09:28:31 dist-test-slave-45dx krb5kdc[23391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008511, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T09:28:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T09:28:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.113.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.113.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.113.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab.
Entry for principal kudu/127.20.113.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab.
I20260806 09:28:31.231490 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786008495763054-20932-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786008495763054-20932-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786008495763054-20932-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.113.62:33715
--webserver_interface=127.20.113.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab
--principal=kudu/127.20.113.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:34459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.113.62:33715
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:28:31.564410 23411 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:28:31.564831 23411 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:28:31.565003 23411 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:28:31.572471 23411 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 09:28:31.572610 23411 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 09:28:31.572700 23411 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:28:31.572799 23411 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:28:31.572888 23411 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 09:28:31.572976 23411 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 09:28:31.583169 23411 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:34459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786008495763054-20932-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786008495763054-20932-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.113.62:33715
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab
--principal=kudu/127.20.113.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.113.62:33715
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.113.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786008495763054-20932-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:28:31.584544 23411 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:28:31.585935 23411 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:28:31.597786 23419 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:28:31.597744 23420 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:28:31.597715 23423 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:28:31.598670 23411 server_base.cc:1034] running on GCE node
I20260806 09:28:31.599465 23411 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:28:31.600886 23411 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:28:31.602133 23411 hybrid_clock.cc:648] HybridClock initialized: now 1786008511602071 us; error 49 us; skew 500 ppm
I20260806 09:28:31.602455 23411 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:28:31 dist-test-slave-45dx krb5kdc[23391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008511, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:28:31.605623 23411 init.cc:377] Logged in from keytab as kudu/127.20.113.62@KRBTEST.COM (short username kudu)
I20260806 09:28:31.606983 23411 webserver.cc:460] Webserver started at http://127.20.113.62:36151/ using document root <none> and password file <none>
I20260806 09:28:31.607626 23411 fs_manager.cc:359] Metadata directory not provided
I20260806 09:28:31.607712 23411 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:28:31.607990 23411 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:28:31.610971 23411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786008495763054-20932-0/minicluster-data/master-0/data/instance:
uuid: "405697530ec54e9eac6662969bfb3711"
format_stamp: "Formatted at 2026-08-06 09:28:31 on dist-test-slave-45dx"
server_key: "c3cff4273d52ec03f4392dd06b6dba5d"
server_key_iv: "612132f456323300cc722be4161e6c7c"
server_key_version: "encryptionkey@0"
I20260806 09:28:31.611707 23411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786008495763054-20932-0/minicluster-data/master-0/wal/instance:
uuid: "405697530ec54e9eac6662969bfb3711"
format_stamp: "Formatted at 2026-08-06 09:28:31 on dist-test-slave-45dx"
server_key: "c3cff4273d52ec03f4392dd06b6dba5d"
server_key_iv: "612132f456323300cc722be4161e6c7c"
server_key_version: "encryptionkey@0"
I20260806 09:28:31.616786 23411 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260806 09:28:31.620127 23430 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:28:31.621628 23411 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 09:28:31.621843 23411 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786008495763054-20932-0/minicluster-data/master-0/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786008495763054-20932-0/minicluster-data/master-0/wal
uuid: "405697530ec54e9eac6662969bfb3711"
format_stamp: "Formatted at 2026-08-06 09:28:31 on dist-test-slave-45dx"
server_key: "c3cff4273d52ec03f4392dd06b6dba5d"
server_key_iv: "612132f456323300cc722be4161e6c7c"
server_key_version: "encryptionkey@0"
I20260806 09:28:31.622156 23411 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786008495763054-20932-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786008495763054-20932-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786008495763054-20932-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:28:31.633565 23411 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:28:31.637765 23411 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:28:31.638017 23411 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:28:31.660619 23411 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.62:33715
I20260806 09:28:31.660660 23515 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.62:33715 every 8 connection(s)
I20260806 09:28:31.662144 23411 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
I20260806 09:28:31.666208 23516 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:28:31.671384 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 23411
I20260806 09:28:31.671612 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786008495763054-20932-0/minicluster-data/master-0/wal/instance
I20260806 09:28:31.672267 20932 external_mini_cluster.cc:1428] Setting key e9e5de0d1778c629de1307fa41479077
I20260806 09:28:31.678423 23516 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 405697530ec54e9eac6662969bfb3711: Bootstrap starting.
I20260806 09:28:31.681731 23516 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 405697530ec54e9eac6662969bfb3711: Neither blocks nor log segments found. Creating new log.
I20260806 09:28:31.682912 23516 log.cc:824] T 00000000000000000000000000000000 P 405697530ec54e9eac6662969bfb3711: Log is configured to *not* fsync() on all Append() calls
I20260806 09:28:31.686367 23516 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 405697530ec54e9eac6662969bfb3711: No bootstrap required, opened a new log
Aug 06 09:28:31 dist-test-slave-45dx krb5kdc[23391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008511, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.113.62@KRBTEST.COM
I20260806 09:28:31.695068 23522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:31.674363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34364 (local address 127.20.113.62:33715)
0806 09:28:31.675151 (+ 788us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:31.675166 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:31.675209 (+ 43us) server_negotiation.cc:407] Connection header received
0806 09:28:31.677016 (+ 1807us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:31.677041 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:31.677403 (+ 362us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:31.677769 (+ 366us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:28:31.679288 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:31.680682 (+ 1394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:31.681926 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:31.682601 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:31.682801 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:31.687672 (+ 4871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:28:31.687679 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:28:31.687693 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:28:31.687731 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:28:31.690955 (+ 3224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:31.691699 (+ 744us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:31.691711 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:31.691718 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:31.691895 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:31.692287 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:31.692296 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:31.692332 (+ 36us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:31.692859 (+ 527us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:28:31.693182 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:31.693646 (+ 464us) server_negotiation.cc:299] Negotiation successful
0806 09:28:31.694067 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":576,"thread_start_us":354,"threads_started":1}
I20260806 09:28:31.694908 23516 raft_consensus.cc:348] T 00000000000000000000000000000000 P 405697530ec54e9eac6662969bfb3711 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "405697530ec54e9eac6662969bfb3711" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 33715 } }
I20260806 09:28:31.695362 23516 raft_consensus.cc:374] T 00000000000000000000000000000000 P 405697530ec54e9eac6662969bfb3711 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:28:31.695499 23516 raft_consensus.cc:729] T 00000000000000000000000000000000 P 405697530ec54e9eac6662969bfb3711 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 405697530ec54e9eac6662969bfb3711, State: Initialized, Role: FOLLOWER
I20260806 09:28:31.696247 23516 consensus_queue.cc:260] T 00000000000000000000000000000000 P 405697530ec54e9eac6662969bfb3711 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "405697530ec54e9eac6662969bfb3711" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 33715 } }
I20260806 09:28:31.696604 23516 raft_consensus.cc:388] T 00000000000000000000000000000000 P 405697530ec54e9eac6662969bfb3711 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 09:28:31.696805 23516 raft_consensus.cc:482] T 00000000000000000000000000000000 P 405697530ec54e9eac6662969bfb3711 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 09:28:31.697014 23516 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 405697530ec54e9eac6662969bfb3711 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:28:31.699719 23516 raft_consensus.cc:504] T 00000000000000000000000000000000 P 405697530ec54e9eac6662969bfb3711 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "405697530ec54e9eac6662969bfb3711" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 33715 } }
I20260806 09:28:31.700237 23516 leader_election.cc:302] T 00000000000000000000000000000000 P 405697530ec54e9eac6662969bfb3711 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 405697530ec54e9eac6662969bfb3711; no voters:
I20260806 09:28:31.700924 23516 leader_election.cc:288] T 00000000000000000000000000000000 P 405697530ec54e9eac6662969bfb3711 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 09:28:31.701014 23525 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 405697530ec54e9eac6662969bfb3711 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:28:31.702877 23525 raft_consensus.cc:686] T 00000000000000000000000000000000 P 405697530ec54e9eac6662969bfb3711 [term 1 LEADER]: Becoming Leader. State: Replica: 405697530ec54e9eac6662969bfb3711, State: Running, Role: LEADER
I20260806 09:28:31.703311 23516 sys_catalog.cc:564] T 00000000000000000000000000000000 P 405697530ec54e9eac6662969bfb3711 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 09:28:31.703459 23525 consensus_queue.cc:237] T 00000000000000000000000000000000 P 405697530ec54e9eac6662969bfb3711 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "405697530ec54e9eac6662969bfb3711" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 33715 } }
I20260806 09:28:31.706321 23528 sys_catalog.cc:455] T 00000000000000000000000000000000 P 405697530ec54e9eac6662969bfb3711 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 405697530ec54e9eac6662969bfb3711. Latest consensus state: current_term: 1 leader_uuid: "405697530ec54e9eac6662969bfb3711" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "405697530ec54e9eac6662969bfb3711" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 33715 } } }
I20260806 09:28:31.706889 23528 sys_catalog.cc:458] T 00000000000000000000000000000000 P 405697530ec54e9eac6662969bfb3711 [sys.catalog]: This master's current role is: LEADER
I20260806 09:28:31.706723 23526 sys_catalog.cc:455] T 00000000000000000000000000000000 P 405697530ec54e9eac6662969bfb3711 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "405697530ec54e9eac6662969bfb3711" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "405697530ec54e9eac6662969bfb3711" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 33715 } } }
I20260806 09:28:31.707382 23526 sys_catalog.cc:458] T 00000000000000000000000000000000 P 405697530ec54e9eac6662969bfb3711 [sys.catalog]: This master's current role is: LEADER
I20260806 09:28:31.709347 23536 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 09:28:31.715219 23536 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 09:28:31.725044 23536 catalog_manager.cc:1347] Generated new cluster ID: 21be64fa7ed64a53bc764864269689ac
I20260806 09:28:31.725168 23536 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 09:28:31.760659 23536 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 09:28:31.762660 23536 catalog_manager.cc:1504] Loading token signing keys...
I20260806 09:28:31.773156 23536 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 405697530ec54e9eac6662969bfb3711: Generated new TSK 0
I20260806 09:28:31.774112 23536 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.113.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.113.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.113.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.1.keytab.
Entry for principal kudu/127.20.113.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.1.keytab.
I20260806 09:28:31.842161 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786008495763054-20932-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786008495763054-20932-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786008495763054-20932-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786008495763054-20932-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.113.1:0
--local_ip_for_outbound_sockets=127.20.113.1
--webserver_interface=127.20.113.1
--webserver_port=0
--tserver_master_addrs=127.20.113.62:33715
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.1.keytab
--principal=kudu/127.20.113.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:34459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:28:32.179666 23551 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:28:32.179976 23551 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:28:32.180158 23551 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:28:32.186993 23551 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:28:32.187134 23551 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:28:32.187279 23551 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 09:28:32.187363 23551 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 09:28:32.187474 23551 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.113.1
I20260806 09:28:32.197006 23551 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:34459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786008495763054-20932-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786008495763054-20932-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.1.keytab
--principal=kudu/127.20.113.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.113.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786008495763054-20932-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.113.1
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.20.113.62:33715
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.113.1
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786008495763054-20932-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:28:32.198267 23551 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:28:32.199566 23551 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:28:32.212596 23557 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:28:32.212628 23558 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:28:32.212548 23561 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:28:32.213593 23551 server_base.cc:1034] running on GCE node
I20260806 09:28:32.214243 23551 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:28:32.215085 23551 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:28:32.216331 23551 hybrid_clock.cc:648] HybridClock initialized: now 1786008512216257 us; error 51 us; skew 500 ppm
I20260806 09:28:32.216651 23551 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:28:32 dist-test-slave-45dx krb5kdc[23391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008512, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:28:32.225819 23551 init.cc:377] Logged in from keytab as kudu/127.20.113.1@KRBTEST.COM (short username kudu)
I20260806 09:28:32.227372 23551 webserver.cc:460] Webserver started at http://127.20.113.1:46019/ using document root <none> and password file <none>
I20260806 09:28:32.228065 23551 fs_manager.cc:359] Metadata directory not provided
I20260806 09:28:32.228206 23551 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:28:32.228596 23551 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:28:32.231556 23551 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786008495763054-20932-0/minicluster-data/ts-0/data/instance:
uuid: "af3e642ec6e14f248dd046cf4cecafd3"
format_stamp: "Formatted at 2026-08-06 09:28:32 on dist-test-slave-45dx"
server_key: "6c0234ce2670a6303b4c0261171dd28f"
server_key_iv: "a19b82684e9a2ef7ab0dd04207589481"
server_key_version: "encryptionkey@0"
I20260806 09:28:32.232426 23551 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786008495763054-20932-0/minicluster-data/ts-0/wal/instance:
uuid: "af3e642ec6e14f248dd046cf4cecafd3"
format_stamp: "Formatted at 2026-08-06 09:28:32 on dist-test-slave-45dx"
server_key: "6c0234ce2670a6303b4c0261171dd28f"
server_key_iv: "a19b82684e9a2ef7ab0dd04207589481"
server_key_version: "encryptionkey@0"
I20260806 09:28:32.237632 23551 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260806 09:28:32.241179 23568 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:28:32.242544 23551 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 09:28:32.242723 23551 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786008495763054-20932-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786008495763054-20932-0/minicluster-data/ts-0/wal
uuid: "af3e642ec6e14f248dd046cf4cecafd3"
format_stamp: "Formatted at 2026-08-06 09:28:32 on dist-test-slave-45dx"
server_key: "6c0234ce2670a6303b4c0261171dd28f"
server_key_iv: "a19b82684e9a2ef7ab0dd04207589481"
server_key_version: "encryptionkey@0"
I20260806 09:28:32.242952 23551 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786008495763054-20932-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786008495763054-20932-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786008495763054-20932-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:28:32.271044 23551 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:28:32.275409 23551 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:28:32.275732 23551 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:28:32.278379 23551 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:28:32.278522 23551 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:28:32.278645 23551 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:28:32.278749 23551 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 09:28:32 dist-test-slave-45dx krb5kdc[23391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008512, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.1@KRBTEST.COM for kudu/127.20.113.62@KRBTEST.COM
I20260806 09:28:32.302522 23585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:32.279139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.1:45845 (local address 127.20.113.62:33715)
0806 09:28:32.279738 (+ 599us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:32.279756 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:32.281015 (+ 1259us) server_negotiation.cc:407] Connection header received
0806 09:28:32.283124 (+ 2109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:32.283130 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:32.283216 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:32.283341 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:28:32.285750 (+ 2409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:32.286707 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:32.288282 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:32.288852 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:32.289026 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:32.294764 (+ 5738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:28:32.294771 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:28:32.294777 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:28:32.294825 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:28:32.298242 (+ 3417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:32.299209 (+ 967us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:32.299214 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:32.299218 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:32.299382 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:32.300017 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:32.300023 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:32.300026 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:32.300346 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:28:32.300512 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:32.301618 (+ 1106us) server_negotiation.cc:299] Negotiation successful
0806 09:28:32.301955 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":478,"thread_start_us":362,"threads_started":1}
I20260806 09:28:32.303176 23586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:32.279836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:33715 (local address 127.20.113.1:45845)
0806 09:28:32.280813 (+ 977us) negotiation.cc:107] Waiting for socket to connect
0806 09:28:32.280859 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 09:28:32.282847 (+ 1988us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:28:32.283611 (+ 764us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:28:32.283624 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:28:32.284497 (+ 873us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:28:32.285483 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:32.285508 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:32.286861 (+ 1353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:32.286866 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:32.288113 (+ 1247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:32.288126 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:32.289147 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:32.289158 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:32.289584 (+ 426us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:32.290471 (+ 887us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:28:32.290509 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:28:32.294426 (+ 3917us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:28:32.298414 (+ 3988us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:28:32.298419 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:28:32.298430 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:28:32.299065 (+ 635us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:28:32.299512 (+ 447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:28:32.299517 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:28:32.299521 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:28:32.299858 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:28:32.300499 (+ 641us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:28:32.300507 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:28:32.301059 (+ 552us) client_negotiation.cc:769] Sending connection context
0806 09:28:32.301451 (+ 392us) client_negotiation.cc:240] Negotiation successful
0806 09:28:32.301956 (+ 505us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":686,"thread_start_us":456,"threads_started":1}
I20260806 09:28:32.324632 23585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:32.309075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.1:33621 (local address 127.20.113.62:33715)
0806 09:28:32.309283 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:32.309291 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:32.309905 (+ 614us) server_negotiation.cc:407] Connection header received
0806 09:28:32.310043 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:32.310052 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:32.310150 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:32.310273 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:28:32.311240 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:32.312660 (+ 1420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:32.314827 (+ 2167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:32.315537 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:32.315671 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:32.317504 (+ 1833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:28:32.317513 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:28:32.317520 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:28:32.317567 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:28:32.321133 (+ 3566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:32.322081 (+ 948us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:32.322089 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:32.322095 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:32.322282 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:32.322880 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:32.322887 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:32.322892 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:32.323309 (+ 417us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:28:32.323480 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:32.324007 (+ 527us) server_negotiation.cc:299] Negotiation successful
0806 09:28:32.324237 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260806 09:28:32.324791 23670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:32.309095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:33715 (local address 127.20.113.1:33621)
0806 09:28:32.309801 (+ 706us) negotiation.cc:107] Waiting for socket to connect
0806 09:28:32.309832 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0806 09:28:32.309935 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:28:32.310311 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:28:32.310314 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:28:32.310835 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:28:32.311113 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:32.311121 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:32.312857 (+ 1736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:32.312866 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:32.314698 (+ 1832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:32.314710 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:32.315754 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:32.315759 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:32.315891 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:32.316538 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:28:32.316554 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:28:32.317289 (+ 735us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:28:32.321345 (+ 4056us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:28:32.321353 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:28:32.321359 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:28:32.321939 (+ 580us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:28:32.322460 (+ 521us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:28:32.322467 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:28:32.322474 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:28:32.322684 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:28:32.323491 (+ 807us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:28:32.323498 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:28:32.323705 (+ 207us) client_negotiation.cc:769] Sending connection context
0806 09:28:32.323887 (+ 182us) client_negotiation.cc:240] Negotiation successful
0806 09:28:32.324246 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"thread_start_us":404,"threads_started":1}
I20260806 09:28:32.333139 23551 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.1:33057
I20260806 09:28:32.333177 23736 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.1:33057 every 8 connection(s)
I20260806 09:28:32.334362 23551 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786008495763054-20932-0/minicluster-data/ts-0/data/info.pb
I20260806 09:28:32.335395 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 23551
I20260806 09:28:32.335613 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786008495763054-20932-0/minicluster-data/ts-0/wal/instance
I20260806 09:28:32.336498 20932 external_mini_cluster.cc:1428] Setting key 46281ee40c5a8c1a1166284b3d37f8a5
I20260806 09:28:32.336509 23737 heartbeater.cc:347] Connected to a master server at 127.20.113.62:33715
I20260806 09:28:32.336740 23737 heartbeater.cc:464] Registering TS with master...
I20260806 09:28:32.337414 23737 heartbeater.cc:511] Master 127.20.113.62:33715 requested a full tablet report, sending...
I20260806 09:28:32.339465 23448 ts_manager.cc:194] Registered new tserver with Master: af3e642ec6e14f248dd046cf4cecafd3 (127.20.113.1:33057)
I20260806 09:28:32.341054 23448 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.113.1@KRBTEST.COM'} at 127.20.113.1:45845
WARNING: no policy specified for kudu/127.20.113.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.113.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.113.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.2.keytab.
Entry for principal kudu/127.20.113.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.2.keytab.
I20260806 09:28:32.387858 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786008495763054-20932-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786008495763054-20932-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786008495763054-20932-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786008495763054-20932-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.113.2:0
--local_ip_for_outbound_sockets=127.20.113.2
--webserver_interface=127.20.113.2
--webserver_port=0
--tserver_master_addrs=127.20.113.62:33715
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.2.keytab
--principal=kudu/127.20.113.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:34459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:28:32.723023 23744 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:28:32.723297 23744 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:28:32.723497 23744 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:28:32.730670 23744 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:28:32.730818 23744 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:28:32.730980 23744 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 09:28:32.731061 23744 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 09:28:32.731151 23744 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.113.2
I20260806 09:28:32.740852 23744 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:34459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786008495763054-20932-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786008495763054-20932-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.2.keytab
--principal=kudu/127.20.113.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.113.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786008495763054-20932-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.113.2
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.20.113.62:33715
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.113.2
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786008495763054-20932-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:28:32.742122 23744 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:28:32.743417 23744 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:28:32.756750 23754 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:28:32.756831 23757 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:28:32.756785 23761 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:28:32.757929 23744 server_base.cc:1034] running on GCE node
I20260806 09:28:32.758512 23744 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:28:32.759428 23744 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:28:32.760622 23744 hybrid_clock.cc:648] HybridClock initialized: now 1786008512760562 us; error 53 us; skew 500 ppm
I20260806 09:28:32.760984 23744 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:28:32 dist-test-slave-45dx krb5kdc[23391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008512, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:28:32.764045 23744 init.cc:377] Logged in from keytab as kudu/127.20.113.2@KRBTEST.COM (short username kudu)
I20260806 09:28:32.765465 23744 webserver.cc:460] Webserver started at http://127.20.113.2:34331/ using document root <none> and password file <none>
I20260806 09:28:32.766090 23744 fs_manager.cc:359] Metadata directory not provided
I20260806 09:28:32.766194 23744 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:28:32.766476 23744 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:28:32.769430 23744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786008495763054-20932-0/minicluster-data/ts-1/data/instance:
uuid: "9655e6155dd54d529a3d007d1b3cffbe"
format_stamp: "Formatted at 2026-08-06 09:28:32 on dist-test-slave-45dx"
server_key: "cb906d12ebf2a4dd355199a9b55447e8"
server_key_iv: "8c639f86d3eff3bc374b3c3855ed475e"
server_key_version: "encryptionkey@0"
I20260806 09:28:32.770175 23744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786008495763054-20932-0/minicluster-data/ts-1/wal/instance:
uuid: "9655e6155dd54d529a3d007d1b3cffbe"
format_stamp: "Formatted at 2026-08-06 09:28:32 on dist-test-slave-45dx"
server_key: "cb906d12ebf2a4dd355199a9b55447e8"
server_key_iv: "8c639f86d3eff3bc374b3c3855ed475e"
server_key_version: "encryptionkey@0"
I20260806 09:28:32.775415 23744 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260806 09:28:32.779057 23772 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:28:32.780431 23744 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 09:28:32.780604 23744 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786008495763054-20932-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786008495763054-20932-0/minicluster-data/ts-1/wal
uuid: "9655e6155dd54d529a3d007d1b3cffbe"
format_stamp: "Formatted at 2026-08-06 09:28:32 on dist-test-slave-45dx"
server_key: "cb906d12ebf2a4dd355199a9b55447e8"
server_key_iv: "8c639f86d3eff3bc374b3c3855ed475e"
server_key_version: "encryptionkey@0"
I20260806 09:28:32.780804 23744 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786008495763054-20932-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786008495763054-20932-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786008495763054-20932-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:28:32.812139 23744 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:28:32.816747 23744 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:28:32.817072 23744 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:28:32.819875 23744 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:28:32.819969 23744 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:28:32.820134 23744 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:28:32.820264 23744 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 09:28:32 dist-test-slave-45dx krb5kdc[23391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008512, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.2@KRBTEST.COM for kudu/127.20.113.62@KRBTEST.COM
I20260806 09:28:32.843016 23585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:32.820449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.2:56297 (local address 127.20.113.62:33715)
0806 09:28:32.820718 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:32.820728 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:32.822056 (+ 1328us) server_negotiation.cc:407] Connection header received
0806 09:28:32.824179 (+ 2123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:32.824185 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:32.824277 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:32.824446 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:28:32.826536 (+ 2090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:32.827555 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:32.829756 (+ 2201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:32.830647 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:32.830822 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:32.835450 (+ 4628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:28:32.835460 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:28:32.835469 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:28:32.835514 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:28:32.839273 (+ 3759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:32.840273 (+ 1000us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:32.840279 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:32.840282 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:32.840475 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:32.841108 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:32.841111 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:32.841113 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:32.841382 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:28:32.841497 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:32.842588 (+ 1091us) server_negotiation.cc:299] Negotiation successful
0806 09:28:32.842755 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260806 09:28:32.845320 23783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:32.821004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:33715 (local address 127.20.113.2:56297)
0806 09:28:32.821856 (+ 852us) negotiation.cc:107] Waiting for socket to connect
0806 09:28:32.821915 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0806 09:28:32.823826 (+ 1911us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:28:32.824611 (+ 785us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:28:32.824619 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:28:32.825341 (+ 722us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:28:32.826278 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:32.826311 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:32.827721 (+ 1410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:32.827725 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:32.829581 (+ 1856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:32.829593 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:32.830890 (+ 1297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:32.830895 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:32.831070 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:32.831818 (+ 748us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:28:32.831860 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:28:32.835212 (+ 3352us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:28:32.839424 (+ 4212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:28:32.839433 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:28:32.839453 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:28:32.840141 (+ 688us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:28:32.840619 (+ 478us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:28:32.840627 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:28:32.840638 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:28:32.840961 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:28:32.841533 (+ 572us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:28:32.841546 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:28:32.842103 (+ 557us) client_negotiation.cc:769] Sending connection context
0806 09:28:32.842502 (+ 399us) client_negotiation.cc:240] Negotiation successful
0806 09:28:32.843186 (+ 684us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":576,"thread_start_us":380,"threads_started":1}
I20260806 09:28:32.866765 23867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:32.851904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:33715 (local address 127.20.113.2:36221)
0806 09:28:32.852813 (+ 909us) negotiation.cc:107] Waiting for socket to connect
0806 09:28:32.852836 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 09:28:32.853086 (+ 250us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:28:32.853461 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:28:32.853466 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:28:32.853929 (+ 463us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:28:32.854210 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:32.854219 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:32.855545 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:32.855550 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:32.856830 (+ 1280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:32.856845 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:32.857554 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:32.857560 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:32.857722 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:32.858370 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:28:32.858388 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:28:32.859096 (+ 708us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:28:32.863434 (+ 4338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:28:32.863440 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:28:32.863444 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:28:32.863832 (+ 388us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:28:32.864343 (+ 511us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:28:32.864350 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:28:32.864359 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:28:32.864552 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:28:32.865343 (+ 791us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:28:32.865347 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:28:32.865584 (+ 237us) client_negotiation.cc:769] Sending connection context
0806 09:28:32.865782 (+ 198us) client_negotiation.cc:240] Negotiation successful
0806 09:28:32.866275 (+ 493us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":736,"thread_start_us":574,"threads_started":1}
I20260806 09:28:32.866829 23585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:32.851463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.2:36221 (local address 127.20.113.62:33715)
0806 09:28:32.851652 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:32.851660 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:32.852940 (+ 1280us) server_negotiation.cc:407] Connection header received
0806 09:28:32.853229 (+ 289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:32.853234 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:32.853309 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:32.853422 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:28:32.854350 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:32.855410 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:32.856975 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:32.857370 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:32.857507 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:32.859285 (+ 1778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:28:32.859291 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:28:32.859295 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:28:32.859333 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:28:32.863285 (+ 3952us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:32.863954 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:32.863962 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:32.863969 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:32.864165 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:32.864669 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:32.864674 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:32.864677 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:32.865127 (+ 450us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:28:32.865278 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:32.866246 (+ 968us) server_negotiation.cc:299] Negotiation successful
0806 09:28:32.866502 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 09:28:32.878278 23744 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.2:39021
I20260806 09:28:32.878340 23943 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.2:39021 every 8 connection(s)
I20260806 09:28:32.880061 23744 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786008495763054-20932-0/minicluster-data/ts-1/data/info.pb
I20260806 09:28:32.882416 23946 heartbeater.cc:347] Connected to a master server at 127.20.113.62:33715
I20260806 09:28:32.882619 23946 heartbeater.cc:464] Registering TS with master...
I20260806 09:28:32.883229 23946 heartbeater.cc:511] Master 127.20.113.62:33715 requested a full tablet report, sending...
I20260806 09:28:32.884609 23448 ts_manager.cc:194] Registered new tserver with Master: 9655e6155dd54d529a3d007d1b3cffbe (127.20.113.2:39021)
I20260806 09:28:32.885492 23448 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.113.2@KRBTEST.COM'} at 127.20.113.2:56297
I20260806 09:28:32.890399 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 23744
I20260806 09:28:32.890589 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786008495763054-20932-0/minicluster-data/ts-1/wal/instance
I20260806 09:28:32.891111 20932 external_mini_cluster.cc:1428] Setting key e1ba4738c1d88ef71f7bb3839f7e6dc2
WARNING: no policy specified for kudu/127.20.113.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.113.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.113.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.3.keytab.
Entry for principal kudu/127.20.113.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.3.keytab.
I20260806 09:28:32.942713 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786008495763054-20932-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786008495763054-20932-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786008495763054-20932-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786008495763054-20932-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.113.3:0
--local_ip_for_outbound_sockets=127.20.113.3
--webserver_interface=127.20.113.3
--webserver_port=0
--tserver_master_addrs=127.20.113.62:33715
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.3.keytab
--principal=kudu/127.20.113.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:34459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:28:33.271492 23950 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:28:33.271807 23950 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:28:33.271981 23950 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:28:33.279484 23950 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:28:33.279633 23950 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:28:33.279769 23950 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 09:28:33.279855 23950 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 09:28:33.279982 23950 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.113.3
I20260806 09:28:33.289774 23950 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:34459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786008495763054-20932-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786008495763054-20932-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.3.keytab
--principal=kudu/127.20.113.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.113.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786008495763054-20932-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.113.3
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.20.113.62:33715
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.113.3
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786008495763054-20932-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:28:33.291109 23950 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:28:33.292537 23950 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:28:33.305824 23955 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:28:33.306049 23960 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:28:33.306166 23956 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:28:33.307348 23950 server_base.cc:1034] running on GCE node
I20260806 09:28:33.307960 23950 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:28:33.308961 23950 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:28:33.310176 23950 hybrid_clock.cc:648] HybridClock initialized: now 1786008513310116 us; error 54 us; skew 500 ppm
I20260806 09:28:33.310510 23950 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:28:33 dist-test-slave-45dx krb5kdc[23391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008513, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:28:33.313700 23950 init.cc:377] Logged in from keytab as kudu/127.20.113.3@KRBTEST.COM (short username kudu)
I20260806 09:28:33.315189 23950 webserver.cc:460] Webserver started at http://127.20.113.3:34101/ using document root <none> and password file <none>
I20260806 09:28:33.315850 23950 fs_manager.cc:359] Metadata directory not provided
I20260806 09:28:33.315969 23950 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:28:33.316241 23950 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:28:33.319186 23950 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786008495763054-20932-0/minicluster-data/ts-2/data/instance:
uuid: "ff83a0fd6cb3436f8fe060c9ca900644"
format_stamp: "Formatted at 2026-08-06 09:28:33 on dist-test-slave-45dx"
server_key: "290ae79153ce2e2d20c2273a1aef9595"
server_key_iv: "443549b88f988d43d503fa9a47da1d4e"
server_key_version: "encryptionkey@0"
I20260806 09:28:33.319916 23950 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786008495763054-20932-0/minicluster-data/ts-2/wal/instance:
uuid: "ff83a0fd6cb3436f8fe060c9ca900644"
format_stamp: "Formatted at 2026-08-06 09:28:33 on dist-test-slave-45dx"
server_key: "290ae79153ce2e2d20c2273a1aef9595"
server_key_iv: "443549b88f988d43d503fa9a47da1d4e"
server_key_version: "encryptionkey@0"
I20260806 09:28:33.325207 23950 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260806 09:28:33.328822 23968 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:28:33.330250 23950 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260806 09:28:33.330451 23950 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786008495763054-20932-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786008495763054-20932-0/minicluster-data/ts-2/wal
uuid: "ff83a0fd6cb3436f8fe060c9ca900644"
format_stamp: "Formatted at 2026-08-06 09:28:33 on dist-test-slave-45dx"
server_key: "290ae79153ce2e2d20c2273a1aef9595"
server_key_iv: "443549b88f988d43d503fa9a47da1d4e"
server_key_version: "encryptionkey@0"
I20260806 09:28:33.330673 23950 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786008495763054-20932-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786008495763054-20932-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786008495763054-20932-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:28:33.344285 23737 heartbeater.cc:503] Master 127.20.113.62:33715 was elected leader, sending a full tablet report...
I20260806 09:28:33.352244 23950 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:28:33.356104 23950 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:28:33.356442 23950 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:28:33.359041 23950 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:28:33.359176 23950 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:28:33.359368 23950 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:28:33.359483 23950 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 09:28:33 dist-test-slave-45dx krb5kdc[23391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008513, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.3@KRBTEST.COM for kudu/127.20.113.62@KRBTEST.COM
I20260806 09:28:33.382270 23585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:33.359479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.3:48705 (local address 127.20.113.62:33715)
0806 09:28:33.359709 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:33.359719 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:33.361312 (+ 1593us) server_negotiation.cc:407] Connection header received
0806 09:28:33.364371 (+ 3059us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:33.364377 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:33.364470 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:33.364569 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:28:33.366755 (+ 2186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:33.367491 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:33.369774 (+ 2283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:33.370575 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:33.370758 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:33.375251 (+ 4493us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:28:33.375256 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:28:33.375259 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:28:33.375291 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:28:33.378796 (+ 3505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:33.379483 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:33.379493 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:33.379495 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:33.379651 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:33.380101 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:33.380105 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:33.380107 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:33.380418 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:28:33.380568 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:33.381633 (+ 1065us) server_negotiation.cc:299] Negotiation successful
0806 09:28:33.381895 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 09:28:33.383317 23982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:33.360189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:33715 (local address 127.20.113.3:48705)
0806 09:28:33.361143 (+ 954us) negotiation.cc:107] Waiting for socket to connect
0806 09:28:33.361193 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 09:28:33.363988 (+ 2795us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:28:33.364932 (+ 944us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:28:33.364948 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:28:33.365682 (+ 734us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:28:33.366585 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:33.366605 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:33.367644 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:33.367649 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:33.369525 (+ 1876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:33.369541 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:33.370882 (+ 1341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:33.370892 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:33.371223 (+ 331us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:33.371924 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:28:33.371951 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:28:33.374933 (+ 2982us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:28:33.378941 (+ 4008us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:28:33.378950 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:28:33.378963 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:28:33.379361 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:28:33.379763 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:28:33.379768 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:28:33.379772 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:28:33.379976 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:28:33.380549 (+ 573us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:28:33.380557 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:28:33.381107 (+ 550us) client_negotiation.cc:769] Sending connection context
0806 09:28:33.381517 (+ 410us) client_negotiation.cc:240] Negotiation successful
0806 09:28:33.382093 (+ 576us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":625,"thread_start_us":386,"threads_started":1}
I20260806 09:28:33.406646 23585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:33.389739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.3:39785 (local address 127.20.113.62:33715)
0806 09:28:33.389932 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:33.389938 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:33.390650 (+ 712us) server_negotiation.cc:407] Connection header received
0806 09:28:33.390861 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:33.390864 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:33.390921 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:33.390994 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:28:33.392459 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:33.393424 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:33.395823 (+ 2399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:33.396206 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:33.396397 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:33.398975 (+ 2578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:28:33.398981 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:28:33.398985 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:28:33.399021 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:28:33.402903 (+ 3882us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:33.403949 (+ 1046us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:33.403953 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:33.403955 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:33.404140 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:33.404677 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:33.404686 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:33.404690 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:33.405120 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:28:33.405279 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:33.406005 (+ 726us) server_negotiation.cc:299] Negotiation successful
0806 09:28:33.406288 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 09:28:33.406956 24062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:33.389756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:33715 (local address 127.20.113.3:39785)
0806 09:28:33.390528 (+ 772us) negotiation.cc:107] Waiting for socket to connect
0806 09:28:33.390554 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 09:28:33.390719 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:28:33.391069 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:28:33.391076 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:28:33.391802 (+ 726us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:28:33.392262 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:33.392278 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:33.393571 (+ 1293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:33.393580 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:33.395638 (+ 2058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:33.395657 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:33.396469 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:33.396478 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:33.396747 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:33.397561 (+ 814us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:28:33.397581 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:28:33.398786 (+ 1205us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:28:33.403103 (+ 4317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:28:33.403111 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:28:33.403117 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:28:33.403827 (+ 710us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:28:33.404241 (+ 414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:28:33.404249 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:28:33.404255 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:28:33.404498 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:28:33.405360 (+ 862us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:28:33.405367 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:28:33.405637 (+ 270us) client_negotiation.cc:769] Sending connection context
0806 09:28:33.405821 (+ 184us) client_negotiation.cc:240] Negotiation successful
0806 09:28:33.406287 (+ 466us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":570,"thread_start_us":401,"threads_started":1}
I20260806 09:28:33.422297 23950 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.3:34751
I20260806 09:28:33.422363 24148 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.3:34751 every 8 connection(s)
I20260806 09:28:33.423549 23950 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786008495763054-20932-0/minicluster-data/ts-2/data/info.pb
I20260806 09:28:33.425719 24149 heartbeater.cc:347] Connected to a master server at 127.20.113.62:33715
I20260806 09:28:33.425946 24149 heartbeater.cc:464] Registering TS with master...
I20260806 09:28:33.426570 24149 heartbeater.cc:511] Master 127.20.113.62:33715 requested a full tablet report, sending...
I20260806 09:28:33.427906 23448 ts_manager.cc:194] Registered new tserver with Master: ff83a0fd6cb3436f8fe060c9ca900644 (127.20.113.3:34751)
I20260806 09:28:33.428807 23448 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.113.3@KRBTEST.COM'} at 127.20.113.3:48705
I20260806 09:28:33.432410 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 23950
I20260806 09:28:33.432585 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786008495763054-20932-0/minicluster-data/ts-2/wal/instance
I20260806 09:28:33.433032 20932 external_mini_cluster.cc:1428] Setting key 0320cdbb79e404070ae80d1030c5bfbf
I20260806 09:28:33.435858 20932 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 09:28:33 dist-test-slave-45dx krb5kdc[23391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008513, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 09:28:33 dist-test-slave-45dx krb5kdc[23391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008513, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.113.62@KRBTEST.COM
I20260806 09:28:33.494076 23585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:33.480119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38114 (local address 127.20.113.62:33715)
0806 09:28:33.480395 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:33.480404 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:33.480557 (+ 153us) server_negotiation.cc:407] Connection header received
0806 09:28:33.480742 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:33.480746 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:33.480836 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:33.480935 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:28:33.482443 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:33.483163 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:33.484433 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:33.484935 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:33.485029 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:33.488957 (+ 3928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:28:33.488962 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:28:33.488965 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:28:33.488998 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:28:33.491416 (+ 2418us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:33.492030 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:33.492035 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:33.492037 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:33.492167 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:33.492610 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:33.492613 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:33.492616 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:33.492895 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:28:33.493164 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:33.493449 (+ 285us) server_negotiation.cc:299] Negotiation successful
0806 09:28:33.493703 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260806 09:28:33.498049 23448 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38114:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 09:28:33.505167 23448 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 09:28:33.528172 24169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:33.517691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.2:39021 (local address 127.0.0.1:51346)
0806 09:28:33.519114 (+ 1423us) negotiation.cc:107] Waiting for socket to connect
0806 09:28:33.519169 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 09:28:33.519400 (+ 231us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:28:33.520628 (+ 1228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:28:33.520639 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:28:33.520715 (+ 76us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:28:33.521227 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:33.521242 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:33.523112 (+ 1870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:33.523117 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:33.525027 (+ 1910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:33.525044 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:33.527132 (+ 2088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:33.527137 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:33.527336 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:33.527386 (+ 50us) client_negotiation.cc:769] Sending connection context
0806 09:28:33.527512 (+ 126us) client_negotiation.cc:240] Negotiation successful
0806 09:28:33.527712 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1190,"mutex_wait_us":37,"spinlock_wait_cycles":7040,"thread_start_us":1031,"threads_started":1}
I20260806 09:28:33.529073 24174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:33.519046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41522 (local address 127.20.113.1:33057)
0806 09:28:33.521281 (+ 2235us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:33.521291 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:33.521322 (+ 31us) server_negotiation.cc:407] Connection header received
0806 09:28:33.521430 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:33.521437 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:33.521661 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:33.521889 (+ 228us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:28:33.522472 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:33.523908 (+ 1436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:33.526565 (+ 2657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:33.527993 (+ 1428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:33.528143 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:33.528508 (+ 365us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:33.528605 (+ 97us) server_negotiation.cc:299] Negotiation successful
0806 09:28:33.528767 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2081,"thread_start_us":1957,"threads_started":1}
I20260806 09:28:33.529065 24170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:33.517862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51346 (local address 127.20.113.2:39021)
0806 09:28:33.518552 (+ 690us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:33.518562 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:33.519632 (+ 1070us) server_negotiation.cc:407] Connection header received
0806 09:28:33.519753 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:33.519764 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:33.520075 (+ 311us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:33.520286 (+ 211us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:28:33.521491 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:33.522950 (+ 1459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:33.525199 (+ 2249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:33.526667 (+ 1468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:33.526790 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:33.526982 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:33.528638 (+ 1656us) server_negotiation.cc:299] Negotiation successful
0806 09:28:33.528795 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":513,"thread_start_us":578,"threads_started":1}
I20260806 09:28:33.529088 24172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:33.518320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.1:33057 (local address 127.0.0.1:41522)
0806 09:28:33.520572 (+ 2252us) negotiation.cc:107] Waiting for socket to connect
0806 09:28:33.520590 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 09:28:33.520717 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:28:33.521895 (+ 1178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:28:33.521900 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:28:33.521925 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:28:33.522270 (+ 345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:33.522281 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:33.524044 (+ 1763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:33.524050 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:33.526369 (+ 2319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:33.526386 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:33.528243 (+ 1857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:33.528253 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:33.528437 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:33.528470 (+ 33us) client_negotiation.cc:769] Sending connection context
0806 09:28:33.528551 (+ 81us) client_negotiation.cc:240] Negotiation successful
0806 09:28:33.528678 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2123,"mutex_wait_us":3,"spinlock_wait_cycles":18048,"thread_start_us":1946,"threads_started":1}
I20260806 09:28:33.529245 24171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:33.518308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.3:34751 (local address 127.0.0.1:49678)
0806 09:28:33.519794 (+ 1486us) negotiation.cc:107] Waiting for socket to connect
0806 09:28:33.519815 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 09:28:33.519958 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:28:33.522267 (+ 2309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:28:33.522271 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:28:33.522295 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:28:33.522704 (+ 409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:33.522715 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:33.524992 (+ 2277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:33.524999 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:33.526706 (+ 1707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:33.526722 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:33.528279 (+ 1557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:33.528284 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:33.528655 (+ 371us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:33.528683 (+ 28us) client_negotiation.cc:769] Sending connection context
0806 09:28:33.528741 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 09:28:33.528865 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1325,"mutex_wait_us":3,"thread_start_us":1163,"threads_started":1}
I20260806 09:28:33.530371 24175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:33.518993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49678 (local address 127.20.113.3:34751)
0806 09:28:33.521722 (+ 2729us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:33.521730 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:33.521779 (+ 49us) server_negotiation.cc:407] Connection header received
0806 09:28:33.521874 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:33.521880 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:33.522089 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:33.522266 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:28:33.523095 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:33.524844 (+ 1749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:33.526905 (+ 2061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:33.528040 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:33.529557 (+ 1517us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:33.529799 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:33.529872 (+ 73us) server_negotiation.cc:299] Negotiation successful
0806 09:28:33.530038 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2562,"thread_start_us":2381,"threads_started":1}
I20260806 09:28:33.532793 23840 tablet_service.cc:1467] Processing CreateTablet for tablet 8fc6995e60e84a7cb9799c949d5c4d6c (DEFAULT_TABLE table=test-table [id=ae7c59eed72e4483ba42ba0e42d0ea18]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:28:33.532793 23642 tablet_service.cc:1467] Processing CreateTablet for tablet 8fc6995e60e84a7cb9799c949d5c4d6c (DEFAULT_TABLE table=test-table [id=ae7c59eed72e4483ba42ba0e42d0ea18]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:28:33.533663 24043 tablet_service.cc:1467] Processing CreateTablet for tablet 8fc6995e60e84a7cb9799c949d5c4d6c (DEFAULT_TABLE table=test-table [id=ae7c59eed72e4483ba42ba0e42d0ea18]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:28:33.534286 23840 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8fc6995e60e84a7cb9799c949d5c4d6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:28:33.534286 23642 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8fc6995e60e84a7cb9799c949d5c4d6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:28:33.535001 24043 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8fc6995e60e84a7cb9799c949d5c4d6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:28:33.545991 24176 tablet_bootstrap.cc:492] T 8fc6995e60e84a7cb9799c949d5c4d6c P ff83a0fd6cb3436f8fe060c9ca900644: Bootstrap starting.
I20260806 09:28:33.548992 24179 tablet_bootstrap.cc:492] T 8fc6995e60e84a7cb9799c949d5c4d6c P 9655e6155dd54d529a3d007d1b3cffbe: Bootstrap starting.
I20260806 09:28:33.549906 24178 tablet_bootstrap.cc:492] T 8fc6995e60e84a7cb9799c949d5c4d6c P af3e642ec6e14f248dd046cf4cecafd3: Bootstrap starting.
I20260806 09:28:33.550585 24176 tablet_bootstrap.cc:654] T 8fc6995e60e84a7cb9799c949d5c4d6c P ff83a0fd6cb3436f8fe060c9ca900644: Neither blocks nor log segments found. Creating new log.
I20260806 09:28:33.551656 24176 log.cc:824] T 8fc6995e60e84a7cb9799c949d5c4d6c P ff83a0fd6cb3436f8fe060c9ca900644: Log is configured to *not* fsync() on all Append() calls
I20260806 09:28:33.552639 24179 tablet_bootstrap.cc:654] T 8fc6995e60e84a7cb9799c949d5c4d6c P 9655e6155dd54d529a3d007d1b3cffbe: Neither blocks nor log segments found. Creating new log.
I20260806 09:28:33.553395 24178 tablet_bootstrap.cc:654] T 8fc6995e60e84a7cb9799c949d5c4d6c P af3e642ec6e14f248dd046cf4cecafd3: Neither blocks nor log segments found. Creating new log.
I20260806 09:28:33.553982 24179 log.cc:824] T 8fc6995e60e84a7cb9799c949d5c4d6c P 9655e6155dd54d529a3d007d1b3cffbe: Log is configured to *not* fsync() on all Append() calls
I20260806 09:28:33.554589 24176 tablet_bootstrap.cc:492] T 8fc6995e60e84a7cb9799c949d5c4d6c P ff83a0fd6cb3436f8fe060c9ca900644: No bootstrap required, opened a new log
I20260806 09:28:33.554693 24178 log.cc:824] T 8fc6995e60e84a7cb9799c949d5c4d6c P af3e642ec6e14f248dd046cf4cecafd3: Log is configured to *not* fsync() on all Append() calls
I20260806 09:28:33.554873 24176 ts_tablet_manager.cc:1397] T 8fc6995e60e84a7cb9799c949d5c4d6c P ff83a0fd6cb3436f8fe060c9ca900644: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260806 09:28:33.556999 24179 tablet_bootstrap.cc:492] T 8fc6995e60e84a7cb9799c949d5c4d6c P 9655e6155dd54d529a3d007d1b3cffbe: No bootstrap required, opened a new log
I20260806 09:28:33.557291 24179 ts_tablet_manager.cc:1397] T 8fc6995e60e84a7cb9799c949d5c4d6c P 9655e6155dd54d529a3d007d1b3cffbe: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.006s
I20260806 09:28:33.557714 24178 tablet_bootstrap.cc:492] T 8fc6995e60e84a7cb9799c949d5c4d6c P af3e642ec6e14f248dd046cf4cecafd3: No bootstrap required, opened a new log
I20260806 09:28:33.557956 24178 ts_tablet_manager.cc:1397] T 8fc6995e60e84a7cb9799c949d5c4d6c P af3e642ec6e14f248dd046cf4cecafd3: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260806 09:28:33.564618 24179 raft_consensus.cc:348] T 8fc6995e60e84a7cb9799c949d5c4d6c P 9655e6155dd54d529a3d007d1b3cffbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9655e6155dd54d529a3d007d1b3cffbe" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 39021 } } peers { permanent_uuid: "ff83a0fd6cb3436f8fe060c9ca900644" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 34751 } } peers { permanent_uuid: "af3e642ec6e14f248dd046cf4cecafd3" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33057 } }
I20260806 09:28:33.564971 24179 raft_consensus.cc:374] T 8fc6995e60e84a7cb9799c949d5c4d6c P 9655e6155dd54d529a3d007d1b3cffbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:28:33.565073 24179 raft_consensus.cc:729] T 8fc6995e60e84a7cb9799c949d5c4d6c P 9655e6155dd54d529a3d007d1b3cffbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9655e6155dd54d529a3d007d1b3cffbe, State: Initialized, Role: FOLLOWER
I20260806 09:28:33.565292 24176 raft_consensus.cc:348] T 8fc6995e60e84a7cb9799c949d5c4d6c P ff83a0fd6cb3436f8fe060c9ca900644 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9655e6155dd54d529a3d007d1b3cffbe" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 39021 } } peers { permanent_uuid: "ff83a0fd6cb3436f8fe060c9ca900644" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 34751 } } peers { permanent_uuid: "af3e642ec6e14f248dd046cf4cecafd3" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33057 } }
I20260806 09:28:33.565711 24176 raft_consensus.cc:374] T 8fc6995e60e84a7cb9799c949d5c4d6c P ff83a0fd6cb3436f8fe060c9ca900644 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:28:33.565807 24176 raft_consensus.cc:729] T 8fc6995e60e84a7cb9799c949d5c4d6c P ff83a0fd6cb3436f8fe060c9ca900644 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff83a0fd6cb3436f8fe060c9ca900644, State: Initialized, Role: FOLLOWER
I20260806 09:28:33.565703 24179 consensus_queue.cc:260] T 8fc6995e60e84a7cb9799c949d5c4d6c P 9655e6155dd54d529a3d007d1b3cffbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9655e6155dd54d529a3d007d1b3cffbe" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 39021 } } peers { permanent_uuid: "ff83a0fd6cb3436f8fe060c9ca900644" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 34751 } } peers { permanent_uuid: "af3e642ec6e14f248dd046cf4cecafd3" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33057 } }
I20260806 09:28:33.566423 24176 consensus_queue.cc:260] T 8fc6995e60e84a7cb9799c949d5c4d6c P ff83a0fd6cb3436f8fe060c9ca900644 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9655e6155dd54d529a3d007d1b3cffbe" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 39021 } } peers { permanent_uuid: "ff83a0fd6cb3436f8fe060c9ca900644" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 34751 } } peers { permanent_uuid: "af3e642ec6e14f248dd046cf4cecafd3" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33057 } }
I20260806 09:28:33.566802 23946 heartbeater.cc:503] Master 127.20.113.62:33715 was elected leader, sending a full tablet report...
I20260806 09:28:33.567546 24149 heartbeater.cc:503] Master 127.20.113.62:33715 was elected leader, sending a full tablet report...
I20260806 09:28:33.568400 24178 raft_consensus.cc:348] T 8fc6995e60e84a7cb9799c949d5c4d6c P af3e642ec6e14f248dd046cf4cecafd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9655e6155dd54d529a3d007d1b3cffbe" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 39021 } } peers { permanent_uuid: "ff83a0fd6cb3436f8fe060c9ca900644" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 34751 } } peers { permanent_uuid: "af3e642ec6e14f248dd046cf4cecafd3" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33057 } }
I20260806 09:28:33.568806 24178 raft_consensus.cc:374] T 8fc6995e60e84a7cb9799c949d5c4d6c P af3e642ec6e14f248dd046cf4cecafd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:28:33.568934 24178 raft_consensus.cc:729] T 8fc6995e60e84a7cb9799c949d5c4d6c P af3e642ec6e14f248dd046cf4cecafd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af3e642ec6e14f248dd046cf4cecafd3, State: Initialized, Role: FOLLOWER
I20260806 09:28:33.569000 24179 ts_tablet_manager.cc:1428] T 8fc6995e60e84a7cb9799c949d5c4d6c P 9655e6155dd54d529a3d007d1b3cffbe: Time spent starting tablet: real 0.012s user 0.008s sys 0.003s
I20260806 09:28:33.569469 24178 consensus_queue.cc:260] T 8fc6995e60e84a7cb9799c949d5c4d6c P af3e642ec6e14f248dd046cf4cecafd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9655e6155dd54d529a3d007d1b3cffbe" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 39021 } } peers { permanent_uuid: "ff83a0fd6cb3436f8fe060c9ca900644" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 34751 } } peers { permanent_uuid: "af3e642ec6e14f248dd046cf4cecafd3" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33057 } }
I20260806 09:28:33.571211 24178 ts_tablet_manager.cc:1428] T 8fc6995e60e84a7cb9799c949d5c4d6c P af3e642ec6e14f248dd046cf4cecafd3: Time spent starting tablet: real 0.013s user 0.014s sys 0.000s
I20260806 09:28:33.571205 24176 ts_tablet_manager.cc:1428] T 8fc6995e60e84a7cb9799c949d5c4d6c P ff83a0fd6cb3436f8fe060c9ca900644: Time spent starting tablet: real 0.016s user 0.009s sys 0.005s
W20260806 09:28:33.586688 23739 tablet.cc:2369] T 8fc6995e60e84a7cb9799c949d5c4d6c P af3e642ec6e14f248dd046cf4cecafd3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 09:28:33.621436 24185 raft_consensus.cc:482] T 8fc6995e60e84a7cb9799c949d5c4d6c P af3e642ec6e14f248dd046cf4cecafd3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 09:28:33.621762 24185 raft_consensus.cc:504] T 8fc6995e60e84a7cb9799c949d5c4d6c P af3e642ec6e14f248dd046cf4cecafd3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9655e6155dd54d529a3d007d1b3cffbe" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 39021 } } peers { permanent_uuid: "ff83a0fd6cb3436f8fe060c9ca900644" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 34751 } } peers { permanent_uuid: "af3e642ec6e14f248dd046cf4cecafd3" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33057 } }
I20260806 09:28:33.623497 24185 leader_election.cc:288] T 8fc6995e60e84a7cb9799c949d5c4d6c P af3e642ec6e14f248dd046cf4cecafd3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9655e6155dd54d529a3d007d1b3cffbe (127.20.113.2:39021), ff83a0fd6cb3436f8fe060c9ca900644 (127.20.113.3:34751)
I20260806 09:28:33.630597 24175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:33.624077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.1:34477 (local address 127.20.113.3:34751)
0806 09:28:33.624330 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:33.624339 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:33.624954 (+ 615us) server_negotiation.cc:407] Connection header received
0806 09:28:33.625135 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:33.625139 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:33.625196 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:33.625299 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:28:33.626098 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:33.626808 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:33.628783 (+ 1975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:33.629602 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:33.629784 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:33.629859 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:33.630201 (+ 342us) server_negotiation.cc:299] Negotiation successful
0806 09:28:33.630320 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260806 09:28:33.630607 24170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:33.623911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.1:60851 (local address 127.20.113.2:39021)
0806 09:28:33.624159 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:33.624167 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:33.624908 (+ 741us) server_negotiation.cc:407] Connection header received
0806 09:28:33.625114 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:33.625118 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:33.625178 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:33.625242 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:28:33.625952 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:33.626777 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:33.628840 (+ 2063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:33.629586 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:33.629744 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:33.629834 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:33.630174 (+ 340us) server_negotiation.cc:299] Negotiation successful
0806 09:28:33.630297 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260806 09:28:33.630719 24187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:33.623891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.2:39021 (local address 127.20.113.1:60851)
0806 09:28:33.624777 (+ 886us) negotiation.cc:107] Waiting for socket to connect
0806 09:28:33.624797 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 09:28:33.624957 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:28:33.625286 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:28:33.625290 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:28:33.625313 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:28:33.625771 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:33.625806 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:33.626918 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:33.626923 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:33.628668 (+ 1745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:33.628680 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:33.629814 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:33.629820 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:33.629981 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:33.630039 (+ 58us) client_negotiation.cc:769] Sending connection context
0806 09:28:33.630112 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 09:28:33.630254 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":722,"mutex_wait_us":45,"thread_start_us":475,"threads_started":1}
I20260806 09:28:33.630970 24188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:33.623941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.3:34751 (local address 127.20.113.1:34477)
0806 09:28:33.624814 (+ 873us) negotiation.cc:107] Waiting for socket to connect
0806 09:28:33.624835 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 09:28:33.624986 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:28:33.625331 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:28:33.625352 (+ 21us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:28:33.625376 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:28:33.625906 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:33.625922 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:33.626974 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:33.626986 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:33.628627 (+ 1641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:33.628640 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:33.629873 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:33.629880 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:33.630018 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:33.630048 (+ 30us) client_negotiation.cc:769] Sending connection context
0806 09:28:33.630149 (+ 101us) client_negotiation.cc:240] Negotiation successful
0806 09:28:33.630269 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":773,"mutex_wait_us":72,"thread_start_us":566,"threads_started":1}
I20260806 09:28:33.631836 23876 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fc6995e60e84a7cb9799c949d5c4d6c" candidate_uuid: "af3e642ec6e14f248dd046cf4cecafd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9655e6155dd54d529a3d007d1b3cffbe" is_pre_election: true
I20260806 09:28:33.631942 24078 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fc6995e60e84a7cb9799c949d5c4d6c" candidate_uuid: "af3e642ec6e14f248dd046cf4cecafd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff83a0fd6cb3436f8fe060c9ca900644" is_pre_election: true
W20260806 09:28:33.632472 23947 tablet.cc:2369] T 8fc6995e60e84a7cb9799c949d5c4d6c P 9655e6155dd54d529a3d007d1b3cffbe: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 09:28:33.632405 23876 raft_consensus.cc:2454] T 8fc6995e60e84a7cb9799c949d5c4d6c P 9655e6155dd54d529a3d007d1b3cffbe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate af3e642ec6e14f248dd046cf4cecafd3 in term 0.
I20260806 09:28:33.632488 24078 raft_consensus.cc:2454] T 8fc6995e60e84a7cb9799c949d5c4d6c P ff83a0fd6cb3436f8fe060c9ca900644 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate af3e642ec6e14f248dd046cf4cecafd3 in term 0.
I20260806 09:28:33.633108 23570 leader_election.cc:302] T 8fc6995e60e84a7cb9799c949d5c4d6c P af3e642ec6e14f248dd046cf4cecafd3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: af3e642ec6e14f248dd046cf4cecafd3, ff83a0fd6cb3436f8fe060c9ca900644; no voters:
I20260806 09:28:33.633497 24185 raft_consensus.cc:2781] T 8fc6995e60e84a7cb9799c949d5c4d6c P af3e642ec6e14f248dd046cf4cecafd3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 09:28:33.633646 24185 raft_consensus.cc:482] T 8fc6995e60e84a7cb9799c949d5c4d6c P af3e642ec6e14f248dd046cf4cecafd3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 09:28:33.633770 24185 raft_consensus.cc:3037] T 8fc6995e60e84a7cb9799c949d5c4d6c P af3e642ec6e14f248dd046cf4cecafd3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:28:33.636128 24185 raft_consensus.cc:504] T 8fc6995e60e84a7cb9799c949d5c4d6c P af3e642ec6e14f248dd046cf4cecafd3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9655e6155dd54d529a3d007d1b3cffbe" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 39021 } } peers { permanent_uuid: "ff83a0fd6cb3436f8fe060c9ca900644" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 34751 } } peers { permanent_uuid: "af3e642ec6e14f248dd046cf4cecafd3" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33057 } }
I20260806 09:28:33.636791 24185 leader_election.cc:288] T 8fc6995e60e84a7cb9799c949d5c4d6c P af3e642ec6e14f248dd046cf4cecafd3 [CANDIDATE]: Term 1 election: Requested vote from peers 9655e6155dd54d529a3d007d1b3cffbe (127.20.113.2:39021), ff83a0fd6cb3436f8fe060c9ca900644 (127.20.113.3:34751)
I20260806 09:28:33.637166 23876 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fc6995e60e84a7cb9799c949d5c4d6c" candidate_uuid: "af3e642ec6e14f248dd046cf4cecafd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9655e6155dd54d529a3d007d1b3cffbe"
I20260806 09:28:33.637318 24078 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fc6995e60e84a7cb9799c949d5c4d6c" candidate_uuid: "af3e642ec6e14f248dd046cf4cecafd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff83a0fd6cb3436f8fe060c9ca900644"
I20260806 09:28:33.637383 23876 raft_consensus.cc:3037] T 8fc6995e60e84a7cb9799c949d5c4d6c P 9655e6155dd54d529a3d007d1b3cffbe [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:28:33.637528 24078 raft_consensus.cc:3037] T 8fc6995e60e84a7cb9799c949d5c4d6c P ff83a0fd6cb3436f8fe060c9ca900644 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:28:33.640612 23876 raft_consensus.cc:2454] T 8fc6995e60e84a7cb9799c949d5c4d6c P 9655e6155dd54d529a3d007d1b3cffbe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af3e642ec6e14f248dd046cf4cecafd3 in term 1.
I20260806 09:28:33.640775 24078 raft_consensus.cc:2454] T 8fc6995e60e84a7cb9799c949d5c4d6c P ff83a0fd6cb3436f8fe060c9ca900644 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af3e642ec6e14f248dd046cf4cecafd3 in term 1.
I20260806 09:28:33.641063 23571 leader_election.cc:302] T 8fc6995e60e84a7cb9799c949d5c4d6c P af3e642ec6e14f248dd046cf4cecafd3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9655e6155dd54d529a3d007d1b3cffbe, af3e642ec6e14f248dd046cf4cecafd3; no voters:
I20260806 09:28:33.641376 24185 raft_consensus.cc:2781] T 8fc6995e60e84a7cb9799c949d5c4d6c P af3e642ec6e14f248dd046cf4cecafd3 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:28:33.641873 24185 raft_consensus.cc:686] T 8fc6995e60e84a7cb9799c949d5c4d6c P af3e642ec6e14f248dd046cf4cecafd3 [term 1 LEADER]: Becoming Leader. State: Replica: af3e642ec6e14f248dd046cf4cecafd3, State: Running, Role: LEADER
I20260806 09:28:33.642572 24185 consensus_queue.cc:237] T 8fc6995e60e84a7cb9799c949d5c4d6c P af3e642ec6e14f248dd046cf4cecafd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9655e6155dd54d529a3d007d1b3cffbe" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 39021 } } peers { permanent_uuid: "ff83a0fd6cb3436f8fe060c9ca900644" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 34751 } } peers { permanent_uuid: "af3e642ec6e14f248dd046cf4cecafd3" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33057 } }
I20260806 09:28:33.647311 23448 catalog_manager.cc:5495] T 8fc6995e60e84a7cb9799c949d5c4d6c P af3e642ec6e14f248dd046cf4cecafd3 reported cstate change: term changed from 0 to 1, leader changed from <none> to af3e642ec6e14f248dd046cf4cecafd3 (127.20.113.1). New cstate: current_term: 1 leader_uuid: "af3e642ec6e14f248dd046cf4cecafd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9655e6155dd54d529a3d007d1b3cffbe" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 39021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff83a0fd6cb3436f8fe060c9ca900644" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 34751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af3e642ec6e14f248dd046cf4cecafd3" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33057 } health_report { overall_health: HEALTHY } } }
W20260806 09:28:33.675220 24151 tablet.cc:2369] T 8fc6995e60e84a7cb9799c949d5c4d6c P ff83a0fd6cb3436f8fe060c9ca900644: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 06 09:28:33 dist-test-slave-45dx krb5kdc[23391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008511, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.62@KRBTEST.COM for kudu/127.20.113.62@KRBTEST.COM
I20260806 09:28:33.716658 24200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:33.703048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:33715 (local address 127.0.0.1:38126)
0806 09:28:33.703656 (+ 608us) negotiation.cc:107] Waiting for socket to connect
0806 09:28:33.703673 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 09:28:33.703905 (+ 232us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:28:33.704269 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:28:33.704274 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:28:33.705099 (+ 825us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:28:33.705387 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:33.705398 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:33.706430 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:33.706438 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:33.707404 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:33.707415 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:33.708165 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:33.708173 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:33.708400 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:33.709095 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:28:33.709118 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:28:33.711427 (+ 2309us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:28:33.714204 (+ 2777us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:28:33.714218 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:28:33.714225 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:28:33.714660 (+ 435us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:28:33.715040 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:28:33.715044 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:28:33.715048 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:28:33.715172 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:28:33.715709 (+ 537us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:28:33.715715 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:28:33.715885 (+ 170us) client_negotiation.cc:769] Sending connection context
0806 09:28:33.716027 (+ 142us) client_negotiation.cc:240] Negotiation successful
0806 09:28:33.716229 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":320,"threads_started":1}
I20260806 09:28:33.716662 23585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:33.702965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38126 (local address 127.20.113.62:33715)
0806 09:28:33.703148 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:33.703153 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:33.703833 (+ 680us) server_negotiation.cc:407] Connection header received
0806 09:28:33.704032 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:33.704037 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:33.704120 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:33.704228 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:28:33.705547 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:33.706290 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:33.707542 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:33.707977 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:33.708103 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:33.711607 (+ 3504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:28:33.711611 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:28:33.711614 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:28:33.711643 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:28:33.714074 (+ 2431us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:33.714783 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:33.714789 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:33.714793 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:33.714924 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:33.715272 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:33.715279 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:33.715282 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:33.715555 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:28:33.715706 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:33.716120 (+ 414us) server_negotiation.cc:299] Negotiation successful
0806 09:28:33.716412 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":57,"server-negotiator.queue_time_us":61}
I20260806 09:28:33.721483 23448 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.20.113.62@KRBTEST.COM'} at 127.0.0.1:38126:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 09:28:33.722482 23448 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 09:28:33.729022 23642 tablet_service.cc:1467] Processing CreateTablet for tablet 9bb241cba08343bda5961c158f42e0c0 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=05b688211a6b46c29ca1f5e755f39329]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 09:28:33.729194 23840 tablet_service.cc:1467] Processing CreateTablet for tablet 9bb241cba08343bda5961c158f42e0c0 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=05b688211a6b46c29ca1f5e755f39329]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 09:28:33.729465 24043 tablet_service.cc:1467] Processing CreateTablet for tablet 9bb241cba08343bda5961c158f42e0c0 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=05b688211a6b46c29ca1f5e755f39329]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 09:28:33.730082 23642 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9bb241cba08343bda5961c158f42e0c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:28:33.730121 23840 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9bb241cba08343bda5961c158f42e0c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:28:33.730262 24043 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9bb241cba08343bda5961c158f42e0c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:28:33.736197 24178 tablet_bootstrap.cc:492] T 9bb241cba08343bda5961c158f42e0c0 P af3e642ec6e14f248dd046cf4cecafd3: Bootstrap starting.
I20260806 09:28:33.737617 24178 tablet_bootstrap.cc:654] T 9bb241cba08343bda5961c158f42e0c0 P af3e642ec6e14f248dd046cf4cecafd3: Neither blocks nor log segments found. Creating new log.
I20260806 09:28:33.737703 24176 tablet_bootstrap.cc:492] T 9bb241cba08343bda5961c158f42e0c0 P ff83a0fd6cb3436f8fe060c9ca900644: Bootstrap starting.
I20260806 09:28:33.737757 24179 tablet_bootstrap.cc:492] T 9bb241cba08343bda5961c158f42e0c0 P 9655e6155dd54d529a3d007d1b3cffbe: Bootstrap starting.
I20260806 09:28:33.739562 24176 tablet_bootstrap.cc:654] T 9bb241cba08343bda5961c158f42e0c0 P ff83a0fd6cb3436f8fe060c9ca900644: Neither blocks nor log segments found. Creating new log.
I20260806 09:28:33.739655 24179 tablet_bootstrap.cc:654] T 9bb241cba08343bda5961c158f42e0c0 P 9655e6155dd54d529a3d007d1b3cffbe: Neither blocks nor log segments found. Creating new log.
I20260806 09:28:33.740360 24178 tablet_bootstrap.cc:492] T 9bb241cba08343bda5961c158f42e0c0 P af3e642ec6e14f248dd046cf4cecafd3: No bootstrap required, opened a new log
I20260806 09:28:33.740566 24178 ts_tablet_manager.cc:1397] T 9bb241cba08343bda5961c158f42e0c0 P af3e642ec6e14f248dd046cf4cecafd3: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.001s
I20260806 09:28:33.741689 24178 raft_consensus.cc:348] T 9bb241cba08343bda5961c158f42e0c0 P af3e642ec6e14f248dd046cf4cecafd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af3e642ec6e14f248dd046cf4cecafd3" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33057 } } peers { permanent_uuid: "9655e6155dd54d529a3d007d1b3cffbe" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 39021 } } peers { permanent_uuid: "ff83a0fd6cb3436f8fe060c9ca900644" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 34751 } }
I20260806 09:28:33.742002 24178 raft_consensus.cc:374] T 9bb241cba08343bda5961c158f42e0c0 P af3e642ec6e14f248dd046cf4cecafd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:28:33.742115 24178 raft_consensus.cc:729] T 9bb241cba08343bda5961c158f42e0c0 P af3e642ec6e14f248dd046cf4cecafd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af3e642ec6e14f248dd046cf4cecafd3, State: Initialized, Role: FOLLOWER
I20260806 09:28:33.742393 24176 tablet_bootstrap.cc:492] T 9bb241cba08343bda5961c158f42e0c0 P ff83a0fd6cb3436f8fe060c9ca900644: No bootstrap required, opened a new log
I20260806 09:28:33.742568 24176 ts_tablet_manager.cc:1397] T 9bb241cba08343bda5961c158f42e0c0 P ff83a0fd6cb3436f8fe060c9ca900644: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260806 09:28:33.742432 24178 consensus_queue.cc:260] T 9bb241cba08343bda5961c158f42e0c0 P af3e642ec6e14f248dd046cf4cecafd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af3e642ec6e14f248dd046cf4cecafd3" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33057 } } peers { permanent_uuid: "9655e6155dd54d529a3d007d1b3cffbe" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 39021 } } peers { permanent_uuid: "ff83a0fd6cb3436f8fe060c9ca900644" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 34751 } }
I20260806 09:28:33.742659 24179 tablet_bootstrap.cc:492] T 9bb241cba08343bda5961c158f42e0c0 P 9655e6155dd54d529a3d007d1b3cffbe: No bootstrap required, opened a new log
I20260806 09:28:33.742812 24179 ts_tablet_manager.cc:1397] T 9bb241cba08343bda5961c158f42e0c0 P 9655e6155dd54d529a3d007d1b3cffbe: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.003s
I20260806 09:28:33.742957 24185 tablet_replica.cc:439] T 9bb241cba08343bda5961c158f42e0c0 P af3e642ec6e14f248dd046cf4cecafd3: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af3e642ec6e14f248dd046cf4cecafd3" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33057 } } peers { permanent_uuid: "9655e6155dd54d529a3d007d1b3cffbe" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 39021 } } peers { permanent_uuid: "ff83a0fd6cb3436f8fe060c9ca900644" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 34751 } } }
I20260806 09:28:33.743268 24185 tablet_replica.cc:442] T 9bb241cba08343bda5961c158f42e0c0 P af3e642ec6e14f248dd046cf4cecafd3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 09:28:33.743464 24176 raft_consensus.cc:348] T 9bb241cba08343bda5961c158f42e0c0 P ff83a0fd6cb3436f8fe060c9ca900644 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af3e642ec6e14f248dd046cf4cecafd3" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33057 } } peers { permanent_uuid: "9655e6155dd54d529a3d007d1b3cffbe" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 39021 } } peers { permanent_uuid: "ff83a0fd6cb3436f8fe060c9ca900644" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 34751 } }
I20260806 09:28:33.743728 24176 raft_consensus.cc:374] T 9bb241cba08343bda5961c158f42e0c0 P ff83a0fd6cb3436f8fe060c9ca900644 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:28:33.743813 24178 ts_tablet_manager.cc:1428] T 9bb241cba08343bda5961c158f42e0c0 P af3e642ec6e14f248dd046cf4cecafd3: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260806 09:28:33.743829 24176 raft_consensus.cc:729] T 9bb241cba08343bda5961c158f42e0c0 P ff83a0fd6cb3436f8fe060c9ca900644 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff83a0fd6cb3436f8fe060c9ca900644, State: Initialized, Role: FOLLOWER
I20260806 09:28:33.744120 24176 consensus_queue.cc:260] T 9bb241cba08343bda5961c158f42e0c0 P ff83a0fd6cb3436f8fe060c9ca900644 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af3e642ec6e14f248dd046cf4cecafd3" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33057 } } peers { permanent_uuid: "9655e6155dd54d529a3d007d1b3cffbe" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 39021 } } peers { permanent_uuid: "ff83a0fd6cb3436f8fe060c9ca900644" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 34751 } }
I20260806 09:28:33.744746 24176 ts_tablet_manager.cc:1428] T 9bb241cba08343bda5961c158f42e0c0 P ff83a0fd6cb3436f8fe060c9ca900644: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 09:28:33.744840 24179 raft_consensus.cc:348] T 9bb241cba08343bda5961c158f42e0c0 P 9655e6155dd54d529a3d007d1b3cffbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af3e642ec6e14f248dd046cf4cecafd3" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33057 } } peers { permanent_uuid: "9655e6155dd54d529a3d007d1b3cffbe" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 39021 } } peers { permanent_uuid: "ff83a0fd6cb3436f8fe060c9ca900644" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 34751 } }
I20260806 09:28:33.745110 24179 raft_consensus.cc:374] T 9bb241cba08343bda5961c158f42e0c0 P 9655e6155dd54d529a3d007d1b3cffbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:28:33.745206 24179 raft_consensus.cc:729] T 9bb241cba08343bda5961c158f42e0c0 P 9655e6155dd54d529a3d007d1b3cffbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9655e6155dd54d529a3d007d1b3cffbe, State: Initialized, Role: FOLLOWER
I20260806 09:28:33.745540 24179 consensus_queue.cc:260] T 9bb241cba08343bda5961c158f42e0c0 P 9655e6155dd54d529a3d007d1b3cffbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af3e642ec6e14f248dd046cf4cecafd3" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33057 } } peers { permanent_uuid: "9655e6155dd54d529a3d007d1b3cffbe" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 39021 } } peers { permanent_uuid: "ff83a0fd6cb3436f8fe060c9ca900644" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 34751 } }
I20260806 09:28:33.745709 24184 tablet_replica.cc:439] T 9bb241cba08343bda5961c158f42e0c0 P ff83a0fd6cb3436f8fe060c9ca900644: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af3e642ec6e14f248dd046cf4cecafd3" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33057 } } peers { permanent_uuid: "9655e6155dd54d529a3d007d1b3cffbe" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 39021 } } peers { permanent_uuid: "ff83a0fd6cb3436f8fe060c9ca900644" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 34751 } } }
I20260806 09:28:33.746115 24184 tablet_replica.cc:442] T 9bb241cba08343bda5961c158f42e0c0 P ff83a0fd6cb3436f8fe060c9ca900644: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 09:28:33.746033 24183 tablet_replica.cc:439] T 9bb241cba08343bda5961c158f42e0c0 P 9655e6155dd54d529a3d007d1b3cffbe: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af3e642ec6e14f248dd046cf4cecafd3" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33057 } } peers { permanent_uuid: "9655e6155dd54d529a3d007d1b3cffbe" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 39021 } } peers { permanent_uuid: "ff83a0fd6cb3436f8fe060c9ca900644" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 34751 } } }
I20260806 09:28:33.746424 24183 tablet_replica.cc:442] T 9bb241cba08343bda5961c158f42e0c0 P 9655e6155dd54d529a3d007d1b3cffbe: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 09:28:33.747500 24179 ts_tablet_manager.cc:1428] T 9bb241cba08343bda5961c158f42e0c0 P 9655e6155dd54d529a3d007d1b3cffbe: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260806 09:28:33.773797 24185 raft_consensus.cc:482] T 9bb241cba08343bda5961c158f42e0c0 P af3e642ec6e14f248dd046cf4cecafd3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 09:28:33.774060 24185 raft_consensus.cc:504] T 9bb241cba08343bda5961c158f42e0c0 P af3e642ec6e14f248dd046cf4cecafd3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af3e642ec6e14f248dd046cf4cecafd3" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33057 } } peers { permanent_uuid: "9655e6155dd54d529a3d007d1b3cffbe" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 39021 } } peers { permanent_uuid: "ff83a0fd6cb3436f8fe060c9ca900644" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 34751 } }
I20260806 09:28:33.774777 24185 leader_election.cc:288] T 9bb241cba08343bda5961c158f42e0c0 P af3e642ec6e14f248dd046cf4cecafd3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9655e6155dd54d529a3d007d1b3cffbe (127.20.113.2:39021), ff83a0fd6cb3436f8fe060c9ca900644 (127.20.113.3:34751)
I20260806 09:28:33.775318 23876 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb241cba08343bda5961c158f42e0c0" candidate_uuid: "af3e642ec6e14f248dd046cf4cecafd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9655e6155dd54d529a3d007d1b3cffbe" is_pre_election: true
I20260806 09:28:33.775386 24078 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb241cba08343bda5961c158f42e0c0" candidate_uuid: "af3e642ec6e14f248dd046cf4cecafd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff83a0fd6cb3436f8fe060c9ca900644" is_pre_election: true
I20260806 09:28:33.775620 23876 raft_consensus.cc:2454] T 9bb241cba08343bda5961c158f42e0c0 P 9655e6155dd54d529a3d007d1b3cffbe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate af3e642ec6e14f248dd046cf4cecafd3 in term 0.
I20260806 09:28:33.775735 24078 raft_consensus.cc:2454] T 9bb241cba08343bda5961c158f42e0c0 P ff83a0fd6cb3436f8fe060c9ca900644 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate af3e642ec6e14f248dd046cf4cecafd3 in term 0.
I20260806 09:28:33.776067 23571 leader_election.cc:302] T 9bb241cba08343bda5961c158f42e0c0 P af3e642ec6e14f248dd046cf4cecafd3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9655e6155dd54d529a3d007d1b3cffbe, af3e642ec6e14f248dd046cf4cecafd3; no voters:
I20260806 09:28:33.776417 24185 raft_consensus.cc:2781] T 9bb241cba08343bda5961c158f42e0c0 P af3e642ec6e14f248dd046cf4cecafd3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 09:28:33.776531 24185 raft_consensus.cc:482] T 9bb241cba08343bda5961c158f42e0c0 P af3e642ec6e14f248dd046cf4cecafd3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 09:28:33.776645 24185 raft_consensus.cc:3037] T 9bb241cba08343bda5961c158f42e0c0 P af3e642ec6e14f248dd046cf4cecafd3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:28:33.778887 24185 raft_consensus.cc:504] T 9bb241cba08343bda5961c158f42e0c0 P af3e642ec6e14f248dd046cf4cecafd3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af3e642ec6e14f248dd046cf4cecafd3" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33057 } } peers { permanent_uuid: "9655e6155dd54d529a3d007d1b3cffbe" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 39021 } } peers { permanent_uuid: "ff83a0fd6cb3436f8fe060c9ca900644" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 34751 } }
I20260806 09:28:33.779547 24185 leader_election.cc:288] T 9bb241cba08343bda5961c158f42e0c0 P af3e642ec6e14f248dd046cf4cecafd3 [CANDIDATE]: Term 1 election: Requested vote from peers 9655e6155dd54d529a3d007d1b3cffbe (127.20.113.2:39021), ff83a0fd6cb3436f8fe060c9ca900644 (127.20.113.3:34751)
I20260806 09:28:33.780128 23876 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb241cba08343bda5961c158f42e0c0" candidate_uuid: "af3e642ec6e14f248dd046cf4cecafd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9655e6155dd54d529a3d007d1b3cffbe"
I20260806 09:28:33.780128 24078 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb241cba08343bda5961c158f42e0c0" candidate_uuid: "af3e642ec6e14f248dd046cf4cecafd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff83a0fd6cb3436f8fe060c9ca900644"
I20260806 09:28:33.780400 23876 raft_consensus.cc:3037] T 9bb241cba08343bda5961c158f42e0c0 P 9655e6155dd54d529a3d007d1b3cffbe [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:28:33.780418 24078 raft_consensus.cc:3037] T 9bb241cba08343bda5961c158f42e0c0 P ff83a0fd6cb3436f8fe060c9ca900644 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:28:33.783571 24078 raft_consensus.cc:2454] T 9bb241cba08343bda5961c158f42e0c0 P ff83a0fd6cb3436f8fe060c9ca900644 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af3e642ec6e14f248dd046cf4cecafd3 in term 1.
I20260806 09:28:33.783571 23876 raft_consensus.cc:2454] T 9bb241cba08343bda5961c158f42e0c0 P 9655e6155dd54d529a3d007d1b3cffbe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af3e642ec6e14f248dd046cf4cecafd3 in term 1.
I20260806 09:28:33.784044 23571 leader_election.cc:302] T 9bb241cba08343bda5961c158f42e0c0 P af3e642ec6e14f248dd046cf4cecafd3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9655e6155dd54d529a3d007d1b3cffbe, af3e642ec6e14f248dd046cf4cecafd3; no voters:
I20260806 09:28:33.784350 24185 raft_consensus.cc:2781] T 9bb241cba08343bda5961c158f42e0c0 P af3e642ec6e14f248dd046cf4cecafd3 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:28:33.784614 24185 raft_consensus.cc:686] T 9bb241cba08343bda5961c158f42e0c0 P af3e642ec6e14f248dd046cf4cecafd3 [term 1 LEADER]: Becoming Leader. State: Replica: af3e642ec6e14f248dd046cf4cecafd3, State: Running, Role: LEADER
I20260806 09:28:33.784875 24185 consensus_queue.cc:237] T 9bb241cba08343bda5961c158f42e0c0 P af3e642ec6e14f248dd046cf4cecafd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af3e642ec6e14f248dd046cf4cecafd3" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33057 } } peers { permanent_uuid: "9655e6155dd54d529a3d007d1b3cffbe" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 39021 } } peers { permanent_uuid: "ff83a0fd6cb3436f8fe060c9ca900644" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 34751 } }
I20260806 09:28:33.786792 24190 tablet_replica.cc:439] T 9bb241cba08343bda5961c158f42e0c0 P af3e642ec6e14f248dd046cf4cecafd3: TxnStatusTablet state changed. Reason: New leader af3e642ec6e14f248dd046cf4cecafd3. Latest consensus state: current_term: 1 leader_uuid: "af3e642ec6e14f248dd046cf4cecafd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af3e642ec6e14f248dd046cf4cecafd3" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33057 } } peers { permanent_uuid: "9655e6155dd54d529a3d007d1b3cffbe" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 39021 } } peers { permanent_uuid: "ff83a0fd6cb3436f8fe060c9ca900644" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 34751 } } }
I20260806 09:28:33.787197 24190 tablet_replica.cc:442] T 9bb241cba08343bda5961c158f42e0c0 P af3e642ec6e14f248dd046cf4cecafd3: This TxnStatusTablet replica's current role is: LEADER
I20260806 09:28:33.788014 24206 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 09:28:33.788355 24206 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 09:28:33.788272 23447 catalog_manager.cc:5495] T 9bb241cba08343bda5961c158f42e0c0 P af3e642ec6e14f248dd046cf4cecafd3 reported cstate change: term changed from 0 to 1, leader changed from <none> to af3e642ec6e14f248dd046cf4cecafd3 (127.20.113.1). New cstate: current_term: 1 leader_uuid: "af3e642ec6e14f248dd046cf4cecafd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af3e642ec6e14f248dd046cf4cecafd3" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33057 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9655e6155dd54d529a3d007d1b3cffbe" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 39021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff83a0fd6cb3436f8fe060c9ca900644" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 34751 } health_report { overall_health: UNKNOWN } } }
I20260806 09:28:33.813268 24174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:33.806704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41532 (local address 127.20.113.1:33057)
0806 09:28:33.806912 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:33.806921 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:33.807060 (+ 139us) server_negotiation.cc:407] Connection header received
0806 09:28:33.807356 (+ 296us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:33.807360 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:33.807448 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:33.807569 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 09:28:33.808027 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:33.808893 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:33.810901 (+ 2008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:33.811338 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:33.811437 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:33.812052 (+ 615us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 09:28:33.812688 (+ 636us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 09:28:33.812813 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:33.812962 (+ 149us) server_negotiation.cc:299] Negotiation successful
0806 09:28:33.813038 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260806 09:28:33.813452 24200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:33.806696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.1:33057 (local address 127.0.0.1:41532)
0806 09:28:33.806959 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0806 09:28:33.806982 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 09:28:33.807142 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:28:33.807623 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:28:33.807629 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:28:33.807642 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 09:28:33.807859 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:33.807871 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:33.809034 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:33.809040 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:33.810754 (+ 1714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:33.810774 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:33.811498 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:33.811505 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:33.811737 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:33.811816 (+ 79us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 09:28:33.812809 (+ 993us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 09:28:33.812835 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 09:28:33.812933 (+ 98us) client_negotiation.cc:240] Negotiation successful
0806 09:28:33.813106 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260806 09:28:33.822438 24078 raft_consensus.cc:1261] T 9bb241cba08343bda5961c158f42e0c0 P ff83a0fd6cb3436f8fe060c9ca900644 [term 1 FOLLOWER]: Refusing update from remote peer af3e642ec6e14f248dd046cf4cecafd3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 09:28:33.822490 23876 raft_consensus.cc:1261] T 9bb241cba08343bda5961c158f42e0c0 P 9655e6155dd54d529a3d007d1b3cffbe [term 1 FOLLOWER]: Refusing update from remote peer af3e642ec6e14f248dd046cf4cecafd3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 09:28:33.823591 24190 consensus_queue.cc:1035] T 9bb241cba08343bda5961c158f42e0c0 P af3e642ec6e14f248dd046cf4cecafd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff83a0fd6cb3436f8fe060c9ca900644" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 34751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 09:28:33.824004 24185 consensus_queue.cc:1035] T 9bb241cba08343bda5961c158f42e0c0 P af3e642ec6e14f248dd046cf4cecafd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9655e6155dd54d529a3d007d1b3cffbe" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 09:28:33.832808 24184 tablet_replica.cc:439] T 9bb241cba08343bda5961c158f42e0c0 P ff83a0fd6cb3436f8fe060c9ca900644: TxnStatusTablet state changed. Reason: New leader af3e642ec6e14f248dd046cf4cecafd3. Latest consensus state: current_term: 1 leader_uuid: "af3e642ec6e14f248dd046cf4cecafd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af3e642ec6e14f248dd046cf4cecafd3" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33057 } } peers { permanent_uuid: "9655e6155dd54d529a3d007d1b3cffbe" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 39021 } } peers { permanent_uuid: "ff83a0fd6cb3436f8fe060c9ca900644" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 34751 } } }
I20260806 09:28:33.833128 24184 tablet_replica.cc:442] T 9bb241cba08343bda5961c158f42e0c0 P ff83a0fd6cb3436f8fe060c9ca900644: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 09:28:33.835346 24211 mvcc.cc:205] Tried to move back new op lower bound from 7315490872605478912 to 7315490872465362944. Current Snapshot: MvccSnapshot[applied={T|T < 7315490872605478912}]
I20260806 09:28:33.836745 24190 tablet_replica.cc:439] T 9bb241cba08343bda5961c158f42e0c0 P af3e642ec6e14f248dd046cf4cecafd3: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "af3e642ec6e14f248dd046cf4cecafd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af3e642ec6e14f248dd046cf4cecafd3" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33057 } } peers { permanent_uuid: "9655e6155dd54d529a3d007d1b3cffbe" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 39021 } } peers { permanent_uuid: "ff83a0fd6cb3436f8fe060c9ca900644" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 34751 } } }
I20260806 09:28:33.837076 24190 tablet_replica.cc:442] T 9bb241cba08343bda5961c158f42e0c0 P af3e642ec6e14f248dd046cf4cecafd3: This TxnStatusTablet replica's current role is: LEADER
I20260806 09:28:33.837297 24212 mvcc.cc:205] Tried to move back new op lower bound from 7315490872605478912 to 7315490872465362944. Current Snapshot: MvccSnapshot[applied={T|T < 7315490872605478912}]
I20260806 09:28:33.848102 24183 tablet_replica.cc:439] T 9bb241cba08343bda5961c158f42e0c0 P 9655e6155dd54d529a3d007d1b3cffbe: TxnStatusTablet state changed. Reason: New leader af3e642ec6e14f248dd046cf4cecafd3. Latest consensus state: current_term: 1 leader_uuid: "af3e642ec6e14f248dd046cf4cecafd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af3e642ec6e14f248dd046cf4cecafd3" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33057 } } peers { permanent_uuid: "9655e6155dd54d529a3d007d1b3cffbe" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 39021 } } peers { permanent_uuid: "ff83a0fd6cb3436f8fe060c9ca900644" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 34751 } } }
I20260806 09:28:33.848497 24183 tablet_replica.cc:442] T 9bb241cba08343bda5961c158f42e0c0 P 9655e6155dd54d529a3d007d1b3cffbe: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 09:28:33.849629 24184 tablet_replica.cc:439] T 9bb241cba08343bda5961c158f42e0c0 P ff83a0fd6cb3436f8fe060c9ca900644: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "af3e642ec6e14f248dd046cf4cecafd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af3e642ec6e14f248dd046cf4cecafd3" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33057 } } peers { permanent_uuid: "9655e6155dd54d529a3d007d1b3cffbe" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 39021 } } peers { permanent_uuid: "ff83a0fd6cb3436f8fe060c9ca900644" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 34751 } } }
I20260806 09:28:33.849896 24184 tablet_replica.cc:442] T 9bb241cba08343bda5961c158f42e0c0 P ff83a0fd6cb3436f8fe060c9ca900644: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 09:28:33.855896 24185 tablet_replica.cc:439] T 9bb241cba08343bda5961c158f42e0c0 P af3e642ec6e14f248dd046cf4cecafd3: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "af3e642ec6e14f248dd046cf4cecafd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af3e642ec6e14f248dd046cf4cecafd3" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33057 } } peers { permanent_uuid: "9655e6155dd54d529a3d007d1b3cffbe" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 39021 } } peers { permanent_uuid: "ff83a0fd6cb3436f8fe060c9ca900644" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 34751 } } }
I20260806 09:28:33.856261 24185 tablet_replica.cc:442] T 9bb241cba08343bda5961c158f42e0c0 P af3e642ec6e14f248dd046cf4cecafd3: This TxnStatusTablet replica's current role is: LEADER
I20260806 09:28:33.860535 24183 tablet_replica.cc:439] T 9bb241cba08343bda5961c158f42e0c0 P 9655e6155dd54d529a3d007d1b3cffbe: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "af3e642ec6e14f248dd046cf4cecafd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af3e642ec6e14f248dd046cf4cecafd3" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33057 } } peers { permanent_uuid: "9655e6155dd54d529a3d007d1b3cffbe" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 39021 } } peers { permanent_uuid: "ff83a0fd6cb3436f8fe060c9ca900644" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 34751 } } }
I20260806 09:28:33.860833 24183 tablet_replica.cc:442] T 9bb241cba08343bda5961c158f42e0c0 P 9655e6155dd54d529a3d007d1b3cffbe: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 09:28:33.864892 24174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:33.857183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41536 (local address 127.20.113.1:33057)
0806 09:28:33.857388 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:33.857394 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:33.857414 (+ 20us) server_negotiation.cc:407] Connection header received
0806 09:28:33.857489 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:33.857494 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:33.857592 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:33.857703 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 09:28:33.858354 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:33.859595 (+ 1241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:33.861884 (+ 2289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:33.862581 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:33.862705 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:33.863499 (+ 794us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 09:28:33.863671 (+ 172us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 09:28:33.863784 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:33.864393 (+ 609us) server_negotiation.cc:299] Negotiation successful
0806 09:28:33.864505 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 09:28:33.884833 24174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:33.878017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.1:43485 (local address 127.20.113.1:33057)
0806 09:28:33.878229 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:33.878236 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:33.878443 (+ 207us) server_negotiation.cc:407] Connection header received
0806 09:28:33.878714 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:33.878720 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:33.878802 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:33.878893 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 09:28:33.879494 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:33.880250 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:33.882430 (+ 2180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:33.883120 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:33.883235 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:33.884038 (+ 803us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 09:28:33.884156 (+ 118us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 09:28:33.884203 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:33.884390 (+ 187us) server_negotiation.cc:299] Negotiation successful
0806 09:28:33.884455 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260806 09:28:33.884907 24231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:33.877544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.1:33057 (local address 127.20.113.1:43485)
0806 09:28:33.878341 (+ 797us) negotiation.cc:107] Waiting for socket to connect
0806 09:28:33.878366 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 09:28:33.878564 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:28:33.878905 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:28:33.878909 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:28:33.878919 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 09:28:33.879331 (+ 412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:33.879342 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:33.880445 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:33.880453 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:33.882275 (+ 1822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:33.882292 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:33.883435 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:33.883442 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:33.883624 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:33.883703 (+ 79us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 09:28:33.884249 (+ 546us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 09:28:33.884254 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 09:28:33.884359 (+ 105us) client_negotiation.cc:240] Negotiation successful
0806 09:28:33.884479 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":625,"thread_start_us":687,"threads_started":1}
I20260806 09:28:33.907881 23876 raft_consensus.cc:1261] T 8fc6995e60e84a7cb9799c949d5c4d6c P 9655e6155dd54d529a3d007d1b3cffbe [term 1 FOLLOWER]: Refusing update from remote peer af3e642ec6e14f248dd046cf4cecafd3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 09:28:33.908552 24185 consensus_queue.cc:1035] T 8fc6995e60e84a7cb9799c949d5c4d6c P af3e642ec6e14f248dd046cf4cecafd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9655e6155dd54d529a3d007d1b3cffbe" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 09:28:33.908782 24078 raft_consensus.cc:1261] T 8fc6995e60e84a7cb9799c949d5c4d6c P ff83a0fd6cb3436f8fe060c9ca900644 [term 1 FOLLOWER]: Refusing update from remote peer af3e642ec6e14f248dd046cf4cecafd3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 09:28:33.909451 24223 consensus_queue.cc:1035] T 8fc6995e60e84a7cb9799c949d5c4d6c P af3e642ec6e14f248dd046cf4cecafd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff83a0fd6cb3436f8fe060c9ca900644" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 34751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 09:28:33.976388 23447 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38114:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 09:28:33.976706 23447 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38114:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 09:28:33.980517 23447 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 405697530ec54e9eac6662969bfb3711: Sending DeleteTablet for 3 replicas of tablet 8fc6995e60e84a7cb9799c949d5c4d6c
I20260806 09:28:33.981623 23840 tablet_service.cc:1514] Processing DeleteTablet for tablet 8fc6995e60e84a7cb9799c949d5c4d6c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:28:33 UTC) from {username='kudu', principal='kudu/127.20.113.62@KRBTEST.COM'} at 127.0.0.1:51346
I20260806 09:28:33.981741 24043 tablet_service.cc:1514] Processing DeleteTablet for tablet 8fc6995e60e84a7cb9799c949d5c4d6c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:28:33 UTC) from {username='kudu', principal='kudu/127.20.113.62@KRBTEST.COM'} at 127.0.0.1:49678
I20260806 09:28:33.982724 24238 tablet_replica.cc:330] T 8fc6995e60e84a7cb9799c949d5c4d6c P 9655e6155dd54d529a3d007d1b3cffbe: stopping tablet replica
I20260806 09:28:33.983350 24238 raft_consensus.cc:2229] T 8fc6995e60e84a7cb9799c949d5c4d6c P 9655e6155dd54d529a3d007d1b3cffbe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 09:28:33.983570 23447 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38114:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 09:28:33.984122 24238 raft_consensus.cc:2258] T 8fc6995e60e84a7cb9799c949d5c4d6c P 9655e6155dd54d529a3d007d1b3cffbe [term 1 FOLLOWER]: Raft consensus is shut down!
W20260806 09:28:33.984194 23447 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 09:28:33.984546 23642 tablet_service.cc:1514] Processing DeleteTablet for tablet 8fc6995e60e84a7cb9799c949d5c4d6c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:28:33 UTC) from {username='kudu', principal='kudu/127.20.113.62@KRBTEST.COM'} at 127.0.0.1:41522
I20260806 09:28:33.986956 24238 ts_tablet_manager.cc:1905] T 8fc6995e60e84a7cb9799c949d5c4d6c P 9655e6155dd54d529a3d007d1b3cffbe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 09:28:33.991909 23641 tablet_service.cc:1467] Processing CreateTablet for tablet d7ccc83ab93e43b0a0647fa3909d0383 (DEFAULT_TABLE table=test-table [id=354cb336bf53440abbb332fe365f10ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:28:33.992487 23641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d7ccc83ab93e43b0a0647fa3909d0383. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:28:33.996052 24238 ts_tablet_manager.cc:1918] T 8fc6995e60e84a7cb9799c949d5c4d6c P 9655e6155dd54d529a3d007d1b3cffbe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 09:28:33.996851 24238 log.cc:1196] T 8fc6995e60e84a7cb9799c949d5c4d6c P 9655e6155dd54d529a3d007d1b3cffbe: Deleting WAL directory at /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786008495763054-20932-0/minicluster-data/ts-1/wal/wals/8fc6995e60e84a7cb9799c949d5c4d6c
I20260806 09:28:33.997222 23840 tablet_service.cc:1467] Processing CreateTablet for tablet d7ccc83ab93e43b0a0647fa3909d0383 (DEFAULT_TABLE table=test-table [id=354cb336bf53440abbb332fe365f10ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:28:33.997648 23840 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d7ccc83ab93e43b0a0647fa3909d0383. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:28:33.997532 24042 tablet_service.cc:1467] Processing CreateTablet for tablet d7ccc83ab93e43b0a0647fa3909d0383 (DEFAULT_TABLE table=test-table [id=354cb336bf53440abbb332fe365f10ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:28:33.997676 24238 ts_tablet_manager.cc:1939] T 8fc6995e60e84a7cb9799c949d5c4d6c P 9655e6155dd54d529a3d007d1b3cffbe: Deleting consensus metadata
I20260806 09:28:33.998055 24042 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d7ccc83ab93e43b0a0647fa3909d0383. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:28:33.999843 23433 catalog_manager.cc:4841] TS 9655e6155dd54d529a3d007d1b3cffbe (127.20.113.2:39021): tablet 8fc6995e60e84a7cb9799c949d5c4d6c (table test-table [id=ae7c59eed72e4483ba42ba0e42d0ea18]) successfully deleted
I20260806 09:28:34.004354 24240 tablet_replica.cc:330] T 8fc6995e60e84a7cb9799c949d5c4d6c P af3e642ec6e14f248dd046cf4cecafd3: stopping tablet replica
I20260806 09:28:34.005599 24176 tablet_bootstrap.cc:492] T d7ccc83ab93e43b0a0647fa3909d0383 P ff83a0fd6cb3436f8fe060c9ca900644: Bootstrap starting.
I20260806 09:28:34.006170 24240 raft_consensus.cc:2229] T 8fc6995e60e84a7cb9799c949d5c4d6c P af3e642ec6e14f248dd046cf4cecafd3 [term 1 LEADER]: Raft consensus shutting down.
I20260806 09:28:34.007254 24240 raft_consensus.cc:2258] T 8fc6995e60e84a7cb9799c949d5c4d6c P af3e642ec6e14f248dd046cf4cecafd3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 09:28:34.007524 24176 tablet_bootstrap.cc:654] T d7ccc83ab93e43b0a0647fa3909d0383 P ff83a0fd6cb3436f8fe060c9ca900644: Neither blocks nor log segments found. Creating new log.
I20260806 09:28:34.008198 24239 tablet_replica.cc:330] T 8fc6995e60e84a7cb9799c949d5c4d6c P ff83a0fd6cb3436f8fe060c9ca900644: stopping tablet replica
I20260806 09:28:34.009465 24239 raft_consensus.cc:2229] T 8fc6995e60e84a7cb9799c949d5c4d6c P ff83a0fd6cb3436f8fe060c9ca900644 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 09:28:34.010129 24239 raft_consensus.cc:2258] T 8fc6995e60e84a7cb9799c949d5c4d6c P ff83a0fd6cb3436f8fe060c9ca900644 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 09:28:34.010704 24179 tablet_bootstrap.cc:492] T d7ccc83ab93e43b0a0647fa3909d0383 P 9655e6155dd54d529a3d007d1b3cffbe: Bootstrap starting.
I20260806 09:28:34.012450 24179 tablet_bootstrap.cc:654] T d7ccc83ab93e43b0a0647fa3909d0383 P 9655e6155dd54d529a3d007d1b3cffbe: Neither blocks nor log segments found. Creating new log.
I20260806 09:28:34.012953 24239 ts_tablet_manager.cc:1905] T 8fc6995e60e84a7cb9799c949d5c4d6c P ff83a0fd6cb3436f8fe060c9ca900644: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 09:28:34.018646 24178 tablet_bootstrap.cc:492] T d7ccc83ab93e43b0a0647fa3909d0383 P af3e642ec6e14f248dd046cf4cecafd3: Bootstrap starting.
I20260806 09:28:34.019709 24176 tablet_bootstrap.cc:492] T d7ccc83ab93e43b0a0647fa3909d0383 P ff83a0fd6cb3436f8fe060c9ca900644: No bootstrap required, opened a new log
I20260806 09:28:34.019899 24176 ts_tablet_manager.cc:1397] T d7ccc83ab93e43b0a0647fa3909d0383 P ff83a0fd6cb3436f8fe060c9ca900644: Time spent bootstrapping tablet: real 0.014s user 0.001s sys 0.011s
I20260806 09:28:34.020500 24178 tablet_bootstrap.cc:654] T d7ccc83ab93e43b0a0647fa3909d0383 P af3e642ec6e14f248dd046cf4cecafd3: Neither blocks nor log segments found. Creating new log.
I20260806 09:28:34.020798 24176 raft_consensus.cc:348] T d7ccc83ab93e43b0a0647fa3909d0383 P ff83a0fd6cb3436f8fe060c9ca900644 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af3e642ec6e14f248dd046cf4cecafd3" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33057 } } peers { permanent_uuid: "9655e6155dd54d529a3d007d1b3cffbe" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 39021 } } peers { permanent_uuid: "ff83a0fd6cb3436f8fe060c9ca900644" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 34751 } }
I20260806 09:28:34.021052 24176 raft_consensus.cc:374] T d7ccc83ab93e43b0a0647fa3909d0383 P ff83a0fd6cb3436f8fe060c9ca900644 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:28:34.021142 24176 raft_consensus.cc:729] T d7ccc83ab93e43b0a0647fa3909d0383 P ff83a0fd6cb3436f8fe060c9ca900644 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff83a0fd6cb3436f8fe060c9ca900644, State: Initialized, Role: FOLLOWER
I20260806 09:28:34.021471 24176 consensus_queue.cc:260] T d7ccc83ab93e43b0a0647fa3909d0383 P ff83a0fd6cb3436f8fe060c9ca900644 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af3e642ec6e14f248dd046cf4cecafd3" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33057 } } peers { permanent_uuid: "9655e6155dd54d529a3d007d1b3cffbe" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 39021 } } peers { permanent_uuid: "ff83a0fd6cb3436f8fe060c9ca900644" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 34751 } }
I20260806 09:28:34.022671 24239 ts_tablet_manager.cc:1918] T 8fc6995e60e84a7cb9799c949d5c4d6c P ff83a0fd6cb3436f8fe060c9ca900644: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 09:28:34.022900 24239 log.cc:1196] T 8fc6995e60e84a7cb9799c949d5c4d6c P ff83a0fd6cb3436f8fe060c9ca900644: Deleting WAL directory at /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786008495763054-20932-0/minicluster-data/ts-2/wal/wals/8fc6995e60e84a7cb9799c949d5c4d6c
I20260806 09:28:34.023098 24176 ts_tablet_manager.cc:1428] T d7ccc83ab93e43b0a0647fa3909d0383 P ff83a0fd6cb3436f8fe060c9ca900644: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 09:28:34.023527 24239 ts_tablet_manager.cc:1939] T 8fc6995e60e84a7cb9799c949d5c4d6c P ff83a0fd6cb3436f8fe060c9ca900644: Deleting consensus metadata
I20260806 09:28:34.025501 23432 catalog_manager.cc:4841] TS ff83a0fd6cb3436f8fe060c9ca900644 (127.20.113.3:34751): tablet 8fc6995e60e84a7cb9799c949d5c4d6c (table test-table [id=ae7c59eed72e4483ba42ba0e42d0ea18]) successfully deleted
I20260806 09:28:34.030040 24240 ts_tablet_manager.cc:1905] T 8fc6995e60e84a7cb9799c949d5c4d6c P af3e642ec6e14f248dd046cf4cecafd3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 09:28:34.033849 24179 tablet_bootstrap.cc:492] T d7ccc83ab93e43b0a0647fa3909d0383 P 9655e6155dd54d529a3d007d1b3cffbe: No bootstrap required, opened a new log
I20260806 09:28:34.034044 24179 ts_tablet_manager.cc:1397] T d7ccc83ab93e43b0a0647fa3909d0383 P 9655e6155dd54d529a3d007d1b3cffbe: Time spent bootstrapping tablet: real 0.023s user 0.003s sys 0.007s
I20260806 09:28:34.034864 24179 raft_consensus.cc:348] T d7ccc83ab93e43b0a0647fa3909d0383 P 9655e6155dd54d529a3d007d1b3cffbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af3e642ec6e14f248dd046cf4cecafd3" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33057 } } peers { permanent_uuid: "9655e6155dd54d529a3d007d1b3cffbe" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 39021 } } peers { permanent_uuid: "ff83a0fd6cb3436f8fe060c9ca900644" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 34751 } }
I20260806 09:28:34.035125 24179 raft_consensus.cc:374] T d7ccc83ab93e43b0a0647fa3909d0383 P 9655e6155dd54d529a3d007d1b3cffbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:28:34.035202 24179 raft_consensus.cc:729] T d7ccc83ab93e43b0a0647fa3909d0383 P 9655e6155dd54d529a3d007d1b3cffbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9655e6155dd54d529a3d007d1b3cffbe, State: Initialized, Role: FOLLOWER
I20260806 09:28:34.035579 24179 consensus_queue.cc:260] T d7ccc83ab93e43b0a0647fa3909d0383 P 9655e6155dd54d529a3d007d1b3cffbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af3e642ec6e14f248dd046cf4cecafd3" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33057 } } peers { permanent_uuid: "9655e6155dd54d529a3d007d1b3cffbe" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 39021 } } peers { permanent_uuid: "ff83a0fd6cb3436f8fe060c9ca900644" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 34751 } }
I20260806 09:28:34.037460 24178 tablet_bootstrap.cc:492] T d7ccc83ab93e43b0a0647fa3909d0383 P af3e642ec6e14f248dd046cf4cecafd3: No bootstrap required, opened a new log
I20260806 09:28:34.037415 24240 ts_tablet_manager.cc:1918] T 8fc6995e60e84a7cb9799c949d5c4d6c P af3e642ec6e14f248dd046cf4cecafd3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 09:28:34.037678 24178 ts_tablet_manager.cc:1397] T d7ccc83ab93e43b0a0647fa3909d0383 P af3e642ec6e14f248dd046cf4cecafd3: Time spent bootstrapping tablet: real 0.019s user 0.000s sys 0.008s
I20260806 09:28:34.037675 24240 log.cc:1196] T 8fc6995e60e84a7cb9799c949d5c4d6c P af3e642ec6e14f248dd046cf4cecafd3: Deleting WAL directory at /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786008495763054-20932-0/minicluster-data/ts-0/wal/wals/8fc6995e60e84a7cb9799c949d5c4d6c
I20260806 09:28:34.038396 24240 ts_tablet_manager.cc:1939] T 8fc6995e60e84a7cb9799c949d5c4d6c P af3e642ec6e14f248dd046cf4cecafd3: Deleting consensus metadata
I20260806 09:28:34.038547 24178 raft_consensus.cc:348] T d7ccc83ab93e43b0a0647fa3909d0383 P af3e642ec6e14f248dd046cf4cecafd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af3e642ec6e14f248dd046cf4cecafd3" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33057 } } peers { permanent_uuid: "9655e6155dd54d529a3d007d1b3cffbe" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 39021 } } peers { permanent_uuid: "ff83a0fd6cb3436f8fe060c9ca900644" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 34751 } }
I20260806 09:28:34.038810 24178 raft_consensus.cc:374] T d7ccc83ab93e43b0a0647fa3909d0383 P af3e642ec6e14f248dd046cf4cecafd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:28:34.038928 24178 raft_consensus.cc:729] T d7ccc83ab93e43b0a0647fa3909d0383 P af3e642ec6e14f248dd046cf4cecafd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af3e642ec6e14f248dd046cf4cecafd3, State: Initialized, Role: FOLLOWER
I20260806 09:28:34.039232 24178 consensus_queue.cc:260] T d7ccc83ab93e43b0a0647fa3909d0383 P af3e642ec6e14f248dd046cf4cecafd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af3e642ec6e14f248dd046cf4cecafd3" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33057 } } peers { permanent_uuid: "9655e6155dd54d529a3d007d1b3cffbe" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 39021 } } peers { permanent_uuid: "ff83a0fd6cb3436f8fe060c9ca900644" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 34751 } }
I20260806 09:28:34.040112 24178 ts_tablet_manager.cc:1428] T d7ccc83ab93e43b0a0647fa3909d0383 P af3e642ec6e14f248dd046cf4cecafd3: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260806 09:28:34.040354 23431 catalog_manager.cc:4841] TS af3e642ec6e14f248dd046cf4cecafd3 (127.20.113.1:33057): tablet 8fc6995e60e84a7cb9799c949d5c4d6c (table test-table [id=ae7c59eed72e4483ba42ba0e42d0ea18]) successfully deleted
I20260806 09:28:34.040768 24179 ts_tablet_manager.cc:1428] T d7ccc83ab93e43b0a0647fa3909d0383 P 9655e6155dd54d529a3d007d1b3cffbe: Time spent starting tablet: real 0.007s user 0.001s sys 0.001s
I20260806 09:28:34.134982 24184 raft_consensus.cc:482] T d7ccc83ab93e43b0a0647fa3909d0383 P ff83a0fd6cb3436f8fe060c9ca900644 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 09:28:34.135273 24184 raft_consensus.cc:504] T d7ccc83ab93e43b0a0647fa3909d0383 P ff83a0fd6cb3436f8fe060c9ca900644 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af3e642ec6e14f248dd046cf4cecafd3" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33057 } } peers { permanent_uuid: "9655e6155dd54d529a3d007d1b3cffbe" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 39021 } } peers { permanent_uuid: "ff83a0fd6cb3436f8fe060c9ca900644" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 34751 } }
I20260806 09:28:34.137022 24184 leader_election.cc:288] T d7ccc83ab93e43b0a0647fa3909d0383 P ff83a0fd6cb3436f8fe060c9ca900644 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers af3e642ec6e14f248dd046cf4cecafd3 (127.20.113.1:33057), 9655e6155dd54d529a3d007d1b3cffbe (127.20.113.2:39021)
I20260806 09:28:34.146260 24174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:34.137543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.3:56585 (local address 127.20.113.1:33057)
0806 09:28:34.137736 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:34.137742 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:34.138321 (+ 579us) server_negotiation.cc:407] Connection header received
0806 09:28:34.138652 (+ 331us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:34.138659 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:34.138774 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:34.138862 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:28:34.139721 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:34.140905 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:34.143551 (+ 2646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:34.144972 (+ 1421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:34.145103 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:34.145174 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:34.145773 (+ 599us) server_negotiation.cc:299] Negotiation successful
0806 09:28:34.145937 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 09:28:34.147037 24246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:34.137200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.1:33057 (local address 127.20.113.3:56585)
0806 09:28:34.138214 (+ 1014us) negotiation.cc:107] Waiting for socket to connect
0806 09:28:34.138234 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 09:28:34.138451 (+ 217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:28:34.139145 (+ 694us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:28:34.139151 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:28:34.139182 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:28:34.139505 (+ 323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:34.139520 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:34.141158 (+ 1638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:34.141164 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:34.143358 (+ 2194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:34.143375 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:34.145330 (+ 1955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:34.145336 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:34.145555 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:34.145601 (+ 46us) client_negotiation.cc:769] Sending connection context
0806 09:28:34.146510 (+ 909us) client_negotiation.cc:240] Negotiation successful
0806 09:28:34.146667 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":897,"threads_started":1}
I20260806 09:28:34.149500 23679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7ccc83ab93e43b0a0647fa3909d0383" candidate_uuid: "ff83a0fd6cb3436f8fe060c9ca900644" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af3e642ec6e14f248dd046cf4cecafd3" is_pre_election: true
I20260806 09:28:34.149953 23679 raft_consensus.cc:2454] T d7ccc83ab93e43b0a0647fa3909d0383 P af3e642ec6e14f248dd046cf4cecafd3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff83a0fd6cb3436f8fe060c9ca900644 in term 0.
I20260806 09:28:34.150565 23969 leader_election.cc:302] T d7ccc83ab93e43b0a0647fa3909d0383 P ff83a0fd6cb3436f8fe060c9ca900644 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: af3e642ec6e14f248dd046cf4cecafd3, ff83a0fd6cb3436f8fe060c9ca900644; no voters:
I20260806 09:28:34.151062 24184 raft_consensus.cc:2781] T d7ccc83ab93e43b0a0647fa3909d0383 P ff83a0fd6cb3436f8fe060c9ca900644 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 09:28:34.151212 24184 raft_consensus.cc:482] T d7ccc83ab93e43b0a0647fa3909d0383 P ff83a0fd6cb3436f8fe060c9ca900644 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 09:28:34.151335 24184 raft_consensus.cc:3037] T d7ccc83ab93e43b0a0647fa3909d0383 P ff83a0fd6cb3436f8fe060c9ca900644 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:28:34.154153 24246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:34.137491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.2:39021 (local address 127.20.113.3:52759)
0806 09:28:34.147327 (+ 9836us) negotiation.cc:107] Waiting for socket to connect
0806 09:28:34.147351 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 09:28:34.147465 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:28:34.148520 (+ 1055us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:28:34.148525 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:28:34.148560 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:28:34.148767 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:34.148779 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:34.150521 (+ 1742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:34.150530 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:34.152053 (+ 1523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:34.152066 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:34.153376 (+ 1310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:34.153382 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:34.153592 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:34.153620 (+ 28us) client_negotiation.cc:769] Sending connection context
0806 09:28:34.153721 (+ 101us) client_negotiation.cc:240] Negotiation successful
0806 09:28:34.153842 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9662,"thread_start_us":631,"threads_started":1}
I20260806 09:28:34.154469 24247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:34.137941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.3:52759 (local address 127.20.113.2:39021)
0806 09:28:34.148106 (+ 10165us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:34.148113 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:34.148133 (+ 20us) server_negotiation.cc:407] Connection header received
0806 09:28:34.148213 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:34.148222 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:34.148349 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:34.148943 (+ 594us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:28:34.149124 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:34.150320 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:34.152225 (+ 1905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:34.153177 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:34.153379 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:34.153462 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:34.153758 (+ 296us) server_negotiation.cc:299] Negotiation successful
0806 09:28:34.153905 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10032,"thread_start_us":9908,"threads_started":1}
I20260806 09:28:34.154767 24184 raft_consensus.cc:504] T d7ccc83ab93e43b0a0647fa3909d0383 P ff83a0fd6cb3436f8fe060c9ca900644 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af3e642ec6e14f248dd046cf4cecafd3" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33057 } } peers { permanent_uuid: "9655e6155dd54d529a3d007d1b3cffbe" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 39021 } } peers { permanent_uuid: "ff83a0fd6cb3436f8fe060c9ca900644" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 34751 } }
I20260806 09:28:34.155175 23875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7ccc83ab93e43b0a0647fa3909d0383" candidate_uuid: "ff83a0fd6cb3436f8fe060c9ca900644" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9655e6155dd54d529a3d007d1b3cffbe" is_pre_election: true
I20260806 09:28:34.155400 23875 raft_consensus.cc:2454] T d7ccc83ab93e43b0a0647fa3909d0383 P 9655e6155dd54d529a3d007d1b3cffbe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff83a0fd6cb3436f8fe060c9ca900644 in term 0.
I20260806 09:28:34.155700 24184 leader_election.cc:288] T d7ccc83ab93e43b0a0647fa3909d0383 P ff83a0fd6cb3436f8fe060c9ca900644 [CANDIDATE]: Term 1 election: Requested vote from peers af3e642ec6e14f248dd046cf4cecafd3 (127.20.113.1:33057), 9655e6155dd54d529a3d007d1b3cffbe (127.20.113.2:39021)
I20260806 09:28:34.156023 23679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7ccc83ab93e43b0a0647fa3909d0383" candidate_uuid: "ff83a0fd6cb3436f8fe060c9ca900644" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af3e642ec6e14f248dd046cf4cecafd3"
I20260806 09:28:34.156240 23679 raft_consensus.cc:3037] T d7ccc83ab93e43b0a0647fa3909d0383 P af3e642ec6e14f248dd046cf4cecafd3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:28:34.156383 23875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7ccc83ab93e43b0a0647fa3909d0383" candidate_uuid: "ff83a0fd6cb3436f8fe060c9ca900644" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9655e6155dd54d529a3d007d1b3cffbe"
I20260806 09:28:34.156630 23875 raft_consensus.cc:3037] T d7ccc83ab93e43b0a0647fa3909d0383 P 9655e6155dd54d529a3d007d1b3cffbe [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:28:34.158774 23679 raft_consensus.cc:2454] T d7ccc83ab93e43b0a0647fa3909d0383 P af3e642ec6e14f248dd046cf4cecafd3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff83a0fd6cb3436f8fe060c9ca900644 in term 1.
I20260806 09:28:34.159215 23969 leader_election.cc:302] T d7ccc83ab93e43b0a0647fa3909d0383 P ff83a0fd6cb3436f8fe060c9ca900644 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: af3e642ec6e14f248dd046cf4cecafd3, ff83a0fd6cb3436f8fe060c9ca900644; no voters:
I20260806 09:28:34.159497 24184 raft_consensus.cc:2781] T d7ccc83ab93e43b0a0647fa3909d0383 P ff83a0fd6cb3436f8fe060c9ca900644 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:28:34.159762 23875 raft_consensus.cc:2454] T d7ccc83ab93e43b0a0647fa3909d0383 P 9655e6155dd54d529a3d007d1b3cffbe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff83a0fd6cb3436f8fe060c9ca900644 in term 1.
I20260806 09:28:34.160095 24184 raft_consensus.cc:686] T d7ccc83ab93e43b0a0647fa3909d0383 P ff83a0fd6cb3436f8fe060c9ca900644 [term 1 LEADER]: Becoming Leader. State: Replica: ff83a0fd6cb3436f8fe060c9ca900644, State: Running, Role: LEADER
I20260806 09:28:34.160588 24184 consensus_queue.cc:237] T d7ccc83ab93e43b0a0647fa3909d0383 P ff83a0fd6cb3436f8fe060c9ca900644 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af3e642ec6e14f248dd046cf4cecafd3" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33057 } } peers { permanent_uuid: "9655e6155dd54d529a3d007d1b3cffbe" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 39021 } } peers { permanent_uuid: "ff83a0fd6cb3436f8fe060c9ca900644" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 34751 } }
I20260806 09:28:34.164515 23447 catalog_manager.cc:5495] T d7ccc83ab93e43b0a0647fa3909d0383 P ff83a0fd6cb3436f8fe060c9ca900644 reported cstate change: term changed from 0 to 1, leader changed from <none> to ff83a0fd6cb3436f8fe060c9ca900644 (127.20.113.3). New cstate: current_term: 1 leader_uuid: "ff83a0fd6cb3436f8fe060c9ca900644" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af3e642ec6e14f248dd046cf4cecafd3" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33057 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9655e6155dd54d529a3d007d1b3cffbe" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 39021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff83a0fd6cb3436f8fe060c9ca900644" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 34751 } health_report { overall_health: HEALTHY } } }
I20260806 09:28:34.190939 24251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:34.180292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49692 (local address 127.20.113.3:34751)
0806 09:28:34.184734 (+ 4442us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:34.184738 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:34.184758 (+ 20us) server_negotiation.cc:407] Connection header received
0806 09:28:34.184866 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:34.184871 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:34.185025 (+ 154us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:34.185176 (+ 151us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 09:28:34.185720 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:34.186633 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:34.188524 (+ 1891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:34.188929 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:34.189023 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:34.189549 (+ 526us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 09:28:34.190151 (+ 602us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 09:28:34.190261 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:34.190436 (+ 175us) server_negotiation.cc:299] Negotiation successful
0806 09:28:34.190501 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4311,"thread_start_us":4159,"threads_started":1}
I20260806 09:28:34.209218 24174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:34.202175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.3:45925 (local address 127.20.113.1:33057)
0806 09:28:34.202419 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:34.202428 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:34.203098 (+ 670us) server_negotiation.cc:407] Connection header received
0806 09:28:34.203312 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:34.203317 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:34.203387 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:34.203463 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 09:28:34.204190 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:34.205110 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:34.207267 (+ 2157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:34.207695 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:34.207788 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:34.208512 (+ 724us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 09:28:34.208656 (+ 144us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 09:28:34.208725 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:34.208911 (+ 186us) server_negotiation.cc:299] Negotiation successful
0806 09:28:34.208977 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 09:28:34.209591 24256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:34.202161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.1:33057 (local address 127.20.113.3:45925)
0806 09:28:34.202963 (+ 802us) negotiation.cc:107] Waiting for socket to connect
0806 09:28:34.203011 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 09:28:34.203173 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:28:34.203513 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:28:34.203518 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:28:34.203531 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 09:28:34.203994 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:34.204007 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:34.205243 (+ 1236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:34.205249 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:34.207119 (+ 1870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:34.207135 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:34.207900 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:34.207907 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:34.208104 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:34.208181 (+ 77us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 09:28:34.208762 (+ 581us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 09:28:34.208775 (+ 13us) client_negotiation.cc:769] Sending connection context
0806 09:28:34.208875 (+ 100us) client_negotiation.cc:240] Negotiation successful
0806 09:28:34.209016 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":600,"thread_start_us":413,"threads_started":1}
I20260806 09:28:34.219126 23875 raft_consensus.cc:1261] T d7ccc83ab93e43b0a0647fa3909d0383 P 9655e6155dd54d529a3d007d1b3cffbe [term 1 FOLLOWER]: Refusing update from remote peer ff83a0fd6cb3436f8fe060c9ca900644: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 09:28:34.219609 23679 raft_consensus.cc:1261] T d7ccc83ab93e43b0a0647fa3909d0383 P af3e642ec6e14f248dd046cf4cecafd3 [term 1 FOLLOWER]: Refusing update from remote peer ff83a0fd6cb3436f8fe060c9ca900644: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 09:28:34.219743 24184 consensus_queue.cc:1035] T d7ccc83ab93e43b0a0647fa3909d0383 P ff83a0fd6cb3436f8fe060c9ca900644 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9655e6155dd54d529a3d007d1b3cffbe" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 09:28:34.220732 24184 consensus_queue.cc:1035] T d7ccc83ab93e43b0a0647fa3909d0383 P ff83a0fd6cb3436f8fe060c9ca900644 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af3e642ec6e14f248dd046cf4cecafd3" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33057 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 09:28:34.238845 24213 mvcc.cc:205] Tried to move back new op lower bound from 7315490874235281408 to 7315490874006204416. Current Snapshot: MvccSnapshot[applied={T|T < 7315490874235281408}]
I20260806 09:28:34.253754 24251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:34.248384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.1:51707 (local address 127.20.113.3:34751)
0806 09:28:34.248829 (+ 445us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:34.248835 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:34.248854 (+ 19us) server_negotiation.cc:407] Connection header received
0806 09:28:34.248912 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:34.248916 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:34.248983 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:34.249051 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 09:28:34.249484 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:34.250418 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:34.251880 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:34.252512 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:34.252641 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:34.253020 (+ 379us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 09:28:34.253119 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 09:28:34.253229 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:34.253370 (+ 141us) server_negotiation.cc:299] Negotiation successful
0806 09:28:34.253445 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":326}
I20260806 09:28:34.253813 24231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:34.248017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.3:34751 (local address 127.20.113.1:51707)
0806 09:28:34.248441 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0806 09:28:34.248462 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 09:28:34.248604 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:28:34.249101 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:28:34.249106 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:28:34.249119 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 09:28:34.249324 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:34.249334 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:34.250553 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:34.250559 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:34.251738 (+ 1179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:34.251750 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:34.252666 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:34.252672 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:34.252828 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:34.252864 (+ 36us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 09:28:34.253231 (+ 367us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 09:28:34.253245 (+ 14us) client_negotiation.cc:769] Sending connection context
0806 09:28:34.253346 (+ 101us) client_negotiation.cc:240] Negotiation successful
0806 09:28:34.253488 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252}
W20260806 09:28:34.255265 24212 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260806 09:28:35.872505 23448 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38114:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 09:28:35.872793 23448 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38114:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 09:28:35.876955 23448 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 405697530ec54e9eac6662969bfb3711: Sending DeleteTablet for 3 replicas of tablet d7ccc83ab93e43b0a0647fa3909d0383
I20260806 09:28:35.877872 23840 tablet_service.cc:1514] Processing DeleteTablet for tablet d7ccc83ab93e43b0a0647fa3909d0383 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:28:35 UTC) from {username='kudu', principal='kudu/127.20.113.62@KRBTEST.COM'} at 127.0.0.1:51346
I20260806 09:28:35.877920 23641 tablet_service.cc:1514] Processing DeleteTablet for tablet d7ccc83ab93e43b0a0647fa3909d0383 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:28:35 UTC) from {username='kudu', principal='kudu/127.20.113.62@KRBTEST.COM'} at 127.0.0.1:41522
I20260806 09:28:35.878738 24042 tablet_service.cc:1514] Processing DeleteTablet for tablet d7ccc83ab93e43b0a0647fa3909d0383 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:28:35 UTC) from {username='kudu', principal='kudu/127.20.113.62@KRBTEST.COM'} at 127.0.0.1:49678
I20260806 09:28:35.900806 24302 tablet_replica.cc:330] T d7ccc83ab93e43b0a0647fa3909d0383 P ff83a0fd6cb3436f8fe060c9ca900644: stopping tablet replica
I20260806 09:28:35.901012 24302 raft_consensus.cc:2229] T d7ccc83ab93e43b0a0647fa3909d0383 P ff83a0fd6cb3436f8fe060c9ca900644 [term 1 LEADER]: Raft consensus shutting down.
I20260806 09:28:35.901620 24302 raft_consensus.cc:2258] T d7ccc83ab93e43b0a0647fa3909d0383 P ff83a0fd6cb3436f8fe060c9ca900644 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 09:28:35.902786 24302 ts_tablet_manager.cc:1905] T d7ccc83ab93e43b0a0647fa3909d0383 P ff83a0fd6cb3436f8fe060c9ca900644: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 09:28:35.909549 24300 tablet_replica.cc:330] T d7ccc83ab93e43b0a0647fa3909d0383 P af3e642ec6e14f248dd046cf4cecafd3: stopping tablet replica
I20260806 09:28:35.909785 24300 raft_consensus.cc:2229] T d7ccc83ab93e43b0a0647fa3909d0383 P af3e642ec6e14f248dd046cf4cecafd3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 09:28:35.910107 24300 raft_consensus.cc:2258] T d7ccc83ab93e43b0a0647fa3909d0383 P af3e642ec6e14f248dd046cf4cecafd3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 09:28:35.910367 24299 tablet_replica.cc:330] T d7ccc83ab93e43b0a0647fa3909d0383 P 9655e6155dd54d529a3d007d1b3cffbe: stopping tablet replica
I20260806 09:28:35.910588 24299 raft_consensus.cc:2229] T d7ccc83ab93e43b0a0647fa3909d0383 P 9655e6155dd54d529a3d007d1b3cffbe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 09:28:35.910844 24299 raft_consensus.cc:2258] T d7ccc83ab93e43b0a0647fa3909d0383 P 9655e6155dd54d529a3d007d1b3cffbe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 09:28:35.911181 24300 ts_tablet_manager.cc:1905] T d7ccc83ab93e43b0a0647fa3909d0383 P af3e642ec6e14f248dd046cf4cecafd3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 09:28:35.911768 24302 ts_tablet_manager.cc:1918] T d7ccc83ab93e43b0a0647fa3909d0383 P ff83a0fd6cb3436f8fe060c9ca900644: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 09:28:35.911831 24299 ts_tablet_manager.cc:1905] T d7ccc83ab93e43b0a0647fa3909d0383 P 9655e6155dd54d529a3d007d1b3cffbe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 09:28:35.911921 24302 log.cc:1196] T d7ccc83ab93e43b0a0647fa3909d0383 P ff83a0fd6cb3436f8fe060c9ca900644: Deleting WAL directory at /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786008495763054-20932-0/minicluster-data/ts-2/wal/wals/d7ccc83ab93e43b0a0647fa3909d0383
I20260806 09:28:35.912416 24302 ts_tablet_manager.cc:1939] T d7ccc83ab93e43b0a0647fa3909d0383 P ff83a0fd6cb3436f8fe060c9ca900644: Deleting consensus metadata
I20260806 09:28:35.913429 23432 catalog_manager.cc:4841] TS ff83a0fd6cb3436f8fe060c9ca900644 (127.20.113.3:34751): tablet d7ccc83ab93e43b0a0647fa3909d0383 (table test-table [id=354cb336bf53440abbb332fe365f10ec]) successfully deleted
I20260806 09:28:35.918247 24300 ts_tablet_manager.cc:1918] T d7ccc83ab93e43b0a0647fa3909d0383 P af3e642ec6e14f248dd046cf4cecafd3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 09:28:35.918411 24300 log.cc:1196] T d7ccc83ab93e43b0a0647fa3909d0383 P af3e642ec6e14f248dd046cf4cecafd3: Deleting WAL directory at /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786008495763054-20932-0/minicluster-data/ts-0/wal/wals/d7ccc83ab93e43b0a0647fa3909d0383
I20260806 09:28:35.918529 24299 ts_tablet_manager.cc:1918] T d7ccc83ab93e43b0a0647fa3909d0383 P 9655e6155dd54d529a3d007d1b3cffbe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 09:28:35.918663 24299 log.cc:1196] T d7ccc83ab93e43b0a0647fa3909d0383 P 9655e6155dd54d529a3d007d1b3cffbe: Deleting WAL directory at /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786008495763054-20932-0/minicluster-data/ts-1/wal/wals/d7ccc83ab93e43b0a0647fa3909d0383
I20260806 09:28:35.918900 24300 ts_tablet_manager.cc:1939] T d7ccc83ab93e43b0a0647fa3909d0383 P af3e642ec6e14f248dd046cf4cecafd3: Deleting consensus metadata
I20260806 09:28:35.919107 24299 ts_tablet_manager.cc:1939] T d7ccc83ab93e43b0a0647fa3909d0383 P 9655e6155dd54d529a3d007d1b3cffbe: Deleting consensus metadata
I20260806 09:28:35.920161 23433 catalog_manager.cc:4841] TS 9655e6155dd54d529a3d007d1b3cffbe (127.20.113.2:39021): tablet d7ccc83ab93e43b0a0647fa3909d0383 (table test-table [id=354cb336bf53440abbb332fe365f10ec]) successfully deleted
I20260806 09:28:35.920189 23431 catalog_manager.cc:4841] TS af3e642ec6e14f248dd046cf4cecafd3 (127.20.113.1:33057): tablet d7ccc83ab93e43b0a0647fa3909d0383 (table test-table [id=354cb336bf53440abbb332fe365f10ec]) successfully deleted
Aug 06 09:28:35 dist-test-slave-45dx krb5kdc[23391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008515, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 09:28:35 dist-test-slave-45dx krb5kdc[23391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008515, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.20.113.62@KRBTEST.COM
I20260806 09:28:35.959111 24311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:35.945045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38130 (local address 127.20.113.62:33715)
0806 09:28:35.945739 (+ 694us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:35.945744 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:35.945768 (+ 24us) server_negotiation.cc:407] Connection header received
0806 09:28:35.945860 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:35.945865 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:35.945926 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:35.946012 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:28:35.947623 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:35.948480 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:35.949645 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:35.950056 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:35.950145 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:35.954095 (+ 3950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:28:35.954100 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:28:35.954103 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:28:35.954135 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:28:35.956715 (+ 2580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:35.957481 (+ 766us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:35.957484 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:35.957487 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:35.957607 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:35.957902 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:35.957905 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:35.957907 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:35.958163 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:28:35.958337 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:35.958571 (+ 234us) server_negotiation.cc:299] Negotiation successful
0806 09:28:35.958727 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":580,"thread_start_us":385,"threads_started":1}
I20260806 09:28:35.962555 23448 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38130:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 09:28:35.963207 23448 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 09:28:35.969619 23641 tablet_service.cc:1467] Processing CreateTablet for tablet 130a00fb94bd464c8944a62333f6d11f (DEFAULT_TABLE table=test-table [id=92d1760eb1594f02a47f9cbe548129b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:28:35.969676 23840 tablet_service.cc:1467] Processing CreateTablet for tablet 130a00fb94bd464c8944a62333f6d11f (DEFAULT_TABLE table=test-table [id=92d1760eb1594f02a47f9cbe548129b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:28:35.969816 24042 tablet_service.cc:1467] Processing CreateTablet for tablet 130a00fb94bd464c8944a62333f6d11f (DEFAULT_TABLE table=test-table [id=92d1760eb1594f02a47f9cbe548129b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:28:35.970130 23641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 130a00fb94bd464c8944a62333f6d11f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:28:35.970204 23840 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 130a00fb94bd464c8944a62333f6d11f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:28:35.970285 24042 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 130a00fb94bd464c8944a62333f6d11f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:28:35.977736 24315 tablet_bootstrap.cc:492] T 130a00fb94bd464c8944a62333f6d11f P af3e642ec6e14f248dd046cf4cecafd3: Bootstrap starting.
I20260806 09:28:35.978413 24317 tablet_bootstrap.cc:492] T 130a00fb94bd464c8944a62333f6d11f P ff83a0fd6cb3436f8fe060c9ca900644: Bootstrap starting.
I20260806 09:28:35.979270 24312 tablet_bootstrap.cc:492] T 130a00fb94bd464c8944a62333f6d11f P 9655e6155dd54d529a3d007d1b3cffbe: Bootstrap starting.
I20260806 09:28:35.980571 24317 tablet_bootstrap.cc:654] T 130a00fb94bd464c8944a62333f6d11f P ff83a0fd6cb3436f8fe060c9ca900644: Neither blocks nor log segments found. Creating new log.
I20260806 09:28:35.980618 24315 tablet_bootstrap.cc:654] T 130a00fb94bd464c8944a62333f6d11f P af3e642ec6e14f248dd046cf4cecafd3: Neither blocks nor log segments found. Creating new log.
I20260806 09:28:35.981364 24312 tablet_bootstrap.cc:654] T 130a00fb94bd464c8944a62333f6d11f P 9655e6155dd54d529a3d007d1b3cffbe: Neither blocks nor log segments found. Creating new log.
I20260806 09:28:35.982963 24315 tablet_bootstrap.cc:492] T 130a00fb94bd464c8944a62333f6d11f P af3e642ec6e14f248dd046cf4cecafd3: No bootstrap required, opened a new log
I20260806 09:28:35.983120 24315 ts_tablet_manager.cc:1397] T 130a00fb94bd464c8944a62333f6d11f P af3e642ec6e14f248dd046cf4cecafd3: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 09:28:35.983194 24317 tablet_bootstrap.cc:492] T 130a00fb94bd464c8944a62333f6d11f P ff83a0fd6cb3436f8fe060c9ca900644: No bootstrap required, opened a new log
I20260806 09:28:35.983325 24317 ts_tablet_manager.cc:1397] T 130a00fb94bd464c8944a62333f6d11f P ff83a0fd6cb3436f8fe060c9ca900644: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 09:28:35.984015 24315 raft_consensus.cc:348] T 130a00fb94bd464c8944a62333f6d11f P af3e642ec6e14f248dd046cf4cecafd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af3e642ec6e14f248dd046cf4cecafd3" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33057 } } peers { permanent_uuid: "9655e6155dd54d529a3d007d1b3cffbe" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 39021 } } peers { permanent_uuid: "ff83a0fd6cb3436f8fe060c9ca900644" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 34751 } }
I20260806 09:28:35.984138 24317 raft_consensus.cc:348] T 130a00fb94bd464c8944a62333f6d11f P ff83a0fd6cb3436f8fe060c9ca900644 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af3e642ec6e14f248dd046cf4cecafd3" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33057 } } peers { permanent_uuid: "9655e6155dd54d529a3d007d1b3cffbe" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 39021 } } peers { permanent_uuid: "ff83a0fd6cb3436f8fe060c9ca900644" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 34751 } }
I20260806 09:28:35.984381 24317 raft_consensus.cc:374] T 130a00fb94bd464c8944a62333f6d11f P ff83a0fd6cb3436f8fe060c9ca900644 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:28:35.984376 24315 raft_consensus.cc:374] T 130a00fb94bd464c8944a62333f6d11f P af3e642ec6e14f248dd046cf4cecafd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:28:35.984500 24315 raft_consensus.cc:729] T 130a00fb94bd464c8944a62333f6d11f P af3e642ec6e14f248dd046cf4cecafd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af3e642ec6e14f248dd046cf4cecafd3, State: Initialized, Role: FOLLOWER
I20260806 09:28:35.984529 24317 raft_consensus.cc:729] T 130a00fb94bd464c8944a62333f6d11f P ff83a0fd6cb3436f8fe060c9ca900644 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff83a0fd6cb3436f8fe060c9ca900644, State: Initialized, Role: FOLLOWER
I20260806 09:28:35.984773 24315 consensus_queue.cc:260] T 130a00fb94bd464c8944a62333f6d11f P af3e642ec6e14f248dd046cf4cecafd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af3e642ec6e14f248dd046cf4cecafd3" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33057 } } peers { permanent_uuid: "9655e6155dd54d529a3d007d1b3cffbe" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 39021 } } peers { permanent_uuid: "ff83a0fd6cb3436f8fe060c9ca900644" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 34751 } }
I20260806 09:28:35.984773 24317 consensus_queue.cc:260] T 130a00fb94bd464c8944a62333f6d11f P ff83a0fd6cb3436f8fe060c9ca900644 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af3e642ec6e14f248dd046cf4cecafd3" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33057 } } peers { permanent_uuid: "9655e6155dd54d529a3d007d1b3cffbe" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 39021 } } peers { permanent_uuid: "ff83a0fd6cb3436f8fe060c9ca900644" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 34751 } }
I20260806 09:28:35.985364 24317 ts_tablet_manager.cc:1428] T 130a00fb94bd464c8944a62333f6d11f P ff83a0fd6cb3436f8fe060c9ca900644: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260806 09:28:35.985386 24315 ts_tablet_manager.cc:1428] T 130a00fb94bd464c8944a62333f6d11f P af3e642ec6e14f248dd046cf4cecafd3: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260806 09:28:35.987090 24312 tablet_bootstrap.cc:492] T 130a00fb94bd464c8944a62333f6d11f P 9655e6155dd54d529a3d007d1b3cffbe: No bootstrap required, opened a new log
I20260806 09:28:35.987275 24312 ts_tablet_manager.cc:1397] T 130a00fb94bd464c8944a62333f6d11f P 9655e6155dd54d529a3d007d1b3cffbe: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260806 09:28:35.988174 24312 raft_consensus.cc:348] T 130a00fb94bd464c8944a62333f6d11f P 9655e6155dd54d529a3d007d1b3cffbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af3e642ec6e14f248dd046cf4cecafd3" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33057 } } peers { permanent_uuid: "9655e6155dd54d529a3d007d1b3cffbe" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 39021 } } peers { permanent_uuid: "ff83a0fd6cb3436f8fe060c9ca900644" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 34751 } }
I20260806 09:28:35.988478 24312 raft_consensus.cc:374] T 130a00fb94bd464c8944a62333f6d11f P 9655e6155dd54d529a3d007d1b3cffbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:28:35.988595 24312 raft_consensus.cc:729] T 130a00fb94bd464c8944a62333f6d11f P 9655e6155dd54d529a3d007d1b3cffbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9655e6155dd54d529a3d007d1b3cffbe, State: Initialized, Role: FOLLOWER
I20260806 09:28:35.988854 24312 consensus_queue.cc:260] T 130a00fb94bd464c8944a62333f6d11f P 9655e6155dd54d529a3d007d1b3cffbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af3e642ec6e14f248dd046cf4cecafd3" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33057 } } peers { permanent_uuid: "9655e6155dd54d529a3d007d1b3cffbe" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 39021 } } peers { permanent_uuid: "ff83a0fd6cb3436f8fe060c9ca900644" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 34751 } }
I20260806 09:28:35.989615 24312 ts_tablet_manager.cc:1428] T 130a00fb94bd464c8944a62333f6d11f P 9655e6155dd54d529a3d007d1b3cffbe: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 09:28:36.249984 24277 raft_consensus.cc:482] T 130a00fb94bd464c8944a62333f6d11f P af3e642ec6e14f248dd046cf4cecafd3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 09:28:36.250262 24277 raft_consensus.cc:504] T 130a00fb94bd464c8944a62333f6d11f P af3e642ec6e14f248dd046cf4cecafd3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af3e642ec6e14f248dd046cf4cecafd3" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33057 } } peers { permanent_uuid: "9655e6155dd54d529a3d007d1b3cffbe" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 39021 } } peers { permanent_uuid: "ff83a0fd6cb3436f8fe060c9ca900644" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 34751 } }
I20260806 09:28:36.251138 24277 leader_election.cc:288] T 130a00fb94bd464c8944a62333f6d11f P af3e642ec6e14f248dd046cf4cecafd3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9655e6155dd54d529a3d007d1b3cffbe (127.20.113.2:39021), ff83a0fd6cb3436f8fe060c9ca900644 (127.20.113.3:34751)
I20260806 09:28:36.251726 23874 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "130a00fb94bd464c8944a62333f6d11f" candidate_uuid: "af3e642ec6e14f248dd046cf4cecafd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9655e6155dd54d529a3d007d1b3cffbe" is_pre_election: true
I20260806 09:28:36.251829 24077 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "130a00fb94bd464c8944a62333f6d11f" candidate_uuid: "af3e642ec6e14f248dd046cf4cecafd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff83a0fd6cb3436f8fe060c9ca900644" is_pre_election: true
I20260806 09:28:36.252070 23874 raft_consensus.cc:2454] T 130a00fb94bd464c8944a62333f6d11f P 9655e6155dd54d529a3d007d1b3cffbe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate af3e642ec6e14f248dd046cf4cecafd3 in term 0.
I20260806 09:28:36.252187 24077 raft_consensus.cc:2454] T 130a00fb94bd464c8944a62333f6d11f P ff83a0fd6cb3436f8fe060c9ca900644 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate af3e642ec6e14f248dd046cf4cecafd3 in term 0.
I20260806 09:28:36.252704 23571 leader_election.cc:302] T 130a00fb94bd464c8944a62333f6d11f P af3e642ec6e14f248dd046cf4cecafd3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9655e6155dd54d529a3d007d1b3cffbe, af3e642ec6e14f248dd046cf4cecafd3; no voters:
I20260806 09:28:36.252964 24277 raft_consensus.cc:2781] T 130a00fb94bd464c8944a62333f6d11f P af3e642ec6e14f248dd046cf4cecafd3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 09:28:36.253073 24277 raft_consensus.cc:482] T 130a00fb94bd464c8944a62333f6d11f P af3e642ec6e14f248dd046cf4cecafd3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 09:28:36.253139 24277 raft_consensus.cc:3037] T 130a00fb94bd464c8944a62333f6d11f P af3e642ec6e14f248dd046cf4cecafd3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:28:36.255241 24277 raft_consensus.cc:504] T 130a00fb94bd464c8944a62333f6d11f P af3e642ec6e14f248dd046cf4cecafd3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af3e642ec6e14f248dd046cf4cecafd3" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33057 } } peers { permanent_uuid: "9655e6155dd54d529a3d007d1b3cffbe" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 39021 } } peers { permanent_uuid: "ff83a0fd6cb3436f8fe060c9ca900644" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 34751 } }
I20260806 09:28:36.255736 24277 leader_election.cc:288] T 130a00fb94bd464c8944a62333f6d11f P af3e642ec6e14f248dd046cf4cecafd3 [CANDIDATE]: Term 1 election: Requested vote from peers 9655e6155dd54d529a3d007d1b3cffbe (127.20.113.2:39021), ff83a0fd6cb3436f8fe060c9ca900644 (127.20.113.3:34751)
I20260806 09:28:36.256340 24077 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "130a00fb94bd464c8944a62333f6d11f" candidate_uuid: "af3e642ec6e14f248dd046cf4cecafd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff83a0fd6cb3436f8fe060c9ca900644"
I20260806 09:28:36.256297 23874 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "130a00fb94bd464c8944a62333f6d11f" candidate_uuid: "af3e642ec6e14f248dd046cf4cecafd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9655e6155dd54d529a3d007d1b3cffbe"
I20260806 09:28:36.256546 24077 raft_consensus.cc:3037] T 130a00fb94bd464c8944a62333f6d11f P ff83a0fd6cb3436f8fe060c9ca900644 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:28:36.256561 23874 raft_consensus.cc:3037] T 130a00fb94bd464c8944a62333f6d11f P 9655e6155dd54d529a3d007d1b3cffbe [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:28:36.259764 24077 raft_consensus.cc:2454] T 130a00fb94bd464c8944a62333f6d11f P ff83a0fd6cb3436f8fe060c9ca900644 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af3e642ec6e14f248dd046cf4cecafd3 in term 1.
I20260806 09:28:36.259807 23874 raft_consensus.cc:2454] T 130a00fb94bd464c8944a62333f6d11f P 9655e6155dd54d529a3d007d1b3cffbe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af3e642ec6e14f248dd046cf4cecafd3 in term 1.
I20260806 09:28:36.260241 23570 leader_election.cc:302] T 130a00fb94bd464c8944a62333f6d11f P af3e642ec6e14f248dd046cf4cecafd3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: af3e642ec6e14f248dd046cf4cecafd3, ff83a0fd6cb3436f8fe060c9ca900644; no voters:
I20260806 09:28:36.260602 24277 raft_consensus.cc:2781] T 130a00fb94bd464c8944a62333f6d11f P af3e642ec6e14f248dd046cf4cecafd3 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:28:36.260783 24277 raft_consensus.cc:686] T 130a00fb94bd464c8944a62333f6d11f P af3e642ec6e14f248dd046cf4cecafd3 [term 1 LEADER]: Becoming Leader. State: Replica: af3e642ec6e14f248dd046cf4cecafd3, State: Running, Role: LEADER
I20260806 09:28:36.261137 24277 consensus_queue.cc:237] T 130a00fb94bd464c8944a62333f6d11f P af3e642ec6e14f248dd046cf4cecafd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af3e642ec6e14f248dd046cf4cecafd3" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33057 } } peers { permanent_uuid: "9655e6155dd54d529a3d007d1b3cffbe" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 39021 } } peers { permanent_uuid: "ff83a0fd6cb3436f8fe060c9ca900644" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 34751 } }
I20260806 09:28:36.264947 23447 catalog_manager.cc:5495] T 130a00fb94bd464c8944a62333f6d11f P af3e642ec6e14f248dd046cf4cecafd3 reported cstate change: term changed from 0 to 1, leader changed from <none> to af3e642ec6e14f248dd046cf4cecafd3 (127.20.113.1). New cstate: current_term: 1 leader_uuid: "af3e642ec6e14f248dd046cf4cecafd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af3e642ec6e14f248dd046cf4cecafd3" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33057 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9655e6155dd54d529a3d007d1b3cffbe" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 39021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff83a0fd6cb3436f8fe060c9ca900644" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 34751 } health_report { overall_health: UNKNOWN } } }
I20260806 09:28:36.357445 24329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:36.351450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41550 (local address 127.20.113.1:33057)
0806 09:28:36.352159 (+ 709us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:36.352165 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:36.352185 (+ 20us) server_negotiation.cc:407] Connection header received
0806 09:28:36.352271 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:36.352277 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:36.352375 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:36.352473 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 09:28:36.352823 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:36.354091 (+ 1268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:36.355362 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:36.356147 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:36.356287 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:36.356685 (+ 398us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 09:28:36.356825 (+ 140us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 09:28:36.356907 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:36.357003 (+ 96us) server_negotiation.cc:299] Negotiation successful
0806 09:28:36.357074 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":532,"thread_start_us":377,"threads_started":1}
I20260806 09:28:36.367069 23874 raft_consensus.cc:1261] T 130a00fb94bd464c8944a62333f6d11f P 9655e6155dd54d529a3d007d1b3cffbe [term 1 FOLLOWER]: Refusing update from remote peer af3e642ec6e14f248dd046cf4cecafd3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 09:28:36.367198 24077 raft_consensus.cc:1261] T 130a00fb94bd464c8944a62333f6d11f P ff83a0fd6cb3436f8fe060c9ca900644 [term 1 FOLLOWER]: Refusing update from remote peer af3e642ec6e14f248dd046cf4cecafd3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 09:28:36.367784 24268 consensus_queue.cc:1035] T 130a00fb94bd464c8944a62333f6d11f P af3e642ec6e14f248dd046cf4cecafd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9655e6155dd54d529a3d007d1b3cffbe" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 09:28:36.368134 24331 consensus_queue.cc:1035] T 130a00fb94bd464c8944a62333f6d11f P af3e642ec6e14f248dd046cf4cecafd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff83a0fd6cb3436f8fe060c9ca900644" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 34751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 09:28:36.413673 23448 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38130:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 09:28:36.413966 23448 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38130:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 09:28:36.416981 23448 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 405697530ec54e9eac6662969bfb3711: Sending DeleteTablet for 3 replicas of tablet 130a00fb94bd464c8944a62333f6d11f
I20260806 09:28:36.417725 23641 tablet_service.cc:1514] Processing DeleteTablet for tablet 130a00fb94bd464c8944a62333f6d11f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:28:36 UTC) from {username='kudu', principal='kudu/127.20.113.62@KRBTEST.COM'} at 127.0.0.1:41522
I20260806 09:28:36.417846 23840 tablet_service.cc:1514] Processing DeleteTablet for tablet 130a00fb94bd464c8944a62333f6d11f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:28:36 UTC) from {username='kudu', principal='kudu/127.20.113.62@KRBTEST.COM'} at 127.0.0.1:51346
I20260806 09:28:36.417920 24042 tablet_service.cc:1514] Processing DeleteTablet for tablet 130a00fb94bd464c8944a62333f6d11f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:28:36 UTC) from {username='kudu', principal='kudu/127.20.113.62@KRBTEST.COM'} at 127.0.0.1:49678
I20260806 09:28:36.418051 24300 tablet_replica.cc:330] T 130a00fb94bd464c8944a62333f6d11f P af3e642ec6e14f248dd046cf4cecafd3: stopping tablet replica
I20260806 09:28:36.418131 24299 tablet_replica.cc:330] T 130a00fb94bd464c8944a62333f6d11f P 9655e6155dd54d529a3d007d1b3cffbe: stopping tablet replica
I20260806 09:28:36.418274 24300 raft_consensus.cc:2229] T 130a00fb94bd464c8944a62333f6d11f P af3e642ec6e14f248dd046cf4cecafd3 [term 1 LEADER]: Raft consensus shutting down.
I20260806 09:28:36.418365 24299 raft_consensus.cc:2229] T 130a00fb94bd464c8944a62333f6d11f P 9655e6155dd54d529a3d007d1b3cffbe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 09:28:36.418537 24338 tablet_replica.cc:330] T 130a00fb94bd464c8944a62333f6d11f P ff83a0fd6cb3436f8fe060c9ca900644: stopping tablet replica
I20260806 09:28:36.418581 24299 raft_consensus.cc:2258] T 130a00fb94bd464c8944a62333f6d11f P 9655e6155dd54d529a3d007d1b3cffbe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 09:28:36.418583 24300 raft_consensus.cc:2258] T 130a00fb94bd464c8944a62333f6d11f P af3e642ec6e14f248dd046cf4cecafd3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 09:28:36.418727 24338 raft_consensus.cc:2229] T 130a00fb94bd464c8944a62333f6d11f P ff83a0fd6cb3436f8fe060c9ca900644 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 09:28:36.418974 24338 raft_consensus.cc:2258] T 130a00fb94bd464c8944a62333f6d11f P ff83a0fd6cb3436f8fe060c9ca900644 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 09:28:36.419479 23448 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38130:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 09:28:36.419929 24300 ts_tablet_manager.cc:1905] T 130a00fb94bd464c8944a62333f6d11f P af3e642ec6e14f248dd046cf4cecafd3: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260806 09:28:36.420173 23448 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 09:28:36.420277 24299 ts_tablet_manager.cc:1905] T 130a00fb94bd464c8944a62333f6d11f P 9655e6155dd54d529a3d007d1b3cffbe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 09:28:36.420950 24338 ts_tablet_manager.cc:1905] T 130a00fb94bd464c8944a62333f6d11f P ff83a0fd6cb3436f8fe060c9ca900644: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 09:28:36.426832 24299 ts_tablet_manager.cc:1918] T 130a00fb94bd464c8944a62333f6d11f P 9655e6155dd54d529a3d007d1b3cffbe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 09:28:36.427000 24299 log.cc:1196] T 130a00fb94bd464c8944a62333f6d11f P 9655e6155dd54d529a3d007d1b3cffbe: Deleting WAL directory at /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786008495763054-20932-0/minicluster-data/ts-1/wal/wals/130a00fb94bd464c8944a62333f6d11f
I20260806 09:28:36.427444 24299 ts_tablet_manager.cc:1939] T 130a00fb94bd464c8944a62333f6d11f P 9655e6155dd54d529a3d007d1b3cffbe: Deleting consensus metadata
I20260806 09:28:36.428141 24338 ts_tablet_manager.cc:1918] T 130a00fb94bd464c8944a62333f6d11f P ff83a0fd6cb3436f8fe060c9ca900644: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 09:28:36.428344 24338 log.cc:1196] T 130a00fb94bd464c8944a62333f6d11f P ff83a0fd6cb3436f8fe060c9ca900644: Deleting WAL directory at /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786008495763054-20932-0/minicluster-data/ts-2/wal/wals/130a00fb94bd464c8944a62333f6d11f
I20260806 09:28:36.428519 23433 catalog_manager.cc:4841] TS 9655e6155dd54d529a3d007d1b3cffbe (127.20.113.2:39021): tablet 130a00fb94bd464c8944a62333f6d11f (table test-table [id=92d1760eb1594f02a47f9cbe548129b9]) successfully deleted
I20260806 09:28:36.428864 24338 ts_tablet_manager.cc:1939] T 130a00fb94bd464c8944a62333f6d11f P ff83a0fd6cb3436f8fe060c9ca900644: Deleting consensus metadata
I20260806 09:28:36.429973 23432 catalog_manager.cc:4841] TS ff83a0fd6cb3436f8fe060c9ca900644 (127.20.113.3:34751): tablet 130a00fb94bd464c8944a62333f6d11f (table test-table [id=92d1760eb1594f02a47f9cbe548129b9]) successfully deleted
I20260806 09:28:36.432096 24300 ts_tablet_manager.cc:1918] T 130a00fb94bd464c8944a62333f6d11f P af3e642ec6e14f248dd046cf4cecafd3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 09:28:36.432332 24300 log.cc:1196] T 130a00fb94bd464c8944a62333f6d11f P af3e642ec6e14f248dd046cf4cecafd3: Deleting WAL directory at /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786008495763054-20932-0/minicluster-data/ts-0/wal/wals/130a00fb94bd464c8944a62333f6d11f
I20260806 09:28:36.432384 23840 tablet_service.cc:1467] Processing CreateTablet for tablet 47b56e8d01234329aaad38beacdf0fd4 (DEFAULT_TABLE table=test-table [id=8083197781c8495abe3c97e50b9c28e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:28:36.432683 24042 tablet_service.cc:1467] Processing CreateTablet for tablet 47b56e8d01234329aaad38beacdf0fd4 (DEFAULT_TABLE table=test-table [id=8083197781c8495abe3c97e50b9c28e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:28:36.432890 23840 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 47b56e8d01234329aaad38beacdf0fd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:28:36.432893 23641 tablet_service.cc:1467] Processing CreateTablet for tablet 47b56e8d01234329aaad38beacdf0fd4 (DEFAULT_TABLE table=test-table [id=8083197781c8495abe3c97e50b9c28e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:28:36.432996 24300 ts_tablet_manager.cc:1939] T 130a00fb94bd464c8944a62333f6d11f P af3e642ec6e14f248dd046cf4cecafd3: Deleting consensus metadata
I20260806 09:28:36.433136 24042 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 47b56e8d01234329aaad38beacdf0fd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:28:36.433403 23641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 47b56e8d01234329aaad38beacdf0fd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:28:36.434073 23431 catalog_manager.cc:4841] TS af3e642ec6e14f248dd046cf4cecafd3 (127.20.113.1:33057): tablet 130a00fb94bd464c8944a62333f6d11f (table test-table [id=92d1760eb1594f02a47f9cbe548129b9]) successfully deleted
I20260806 09:28:36.439427 24317 tablet_bootstrap.cc:492] T 47b56e8d01234329aaad38beacdf0fd4 P ff83a0fd6cb3436f8fe060c9ca900644: Bootstrap starting.
I20260806 09:28:36.439424 24312 tablet_bootstrap.cc:492] T 47b56e8d01234329aaad38beacdf0fd4 P 9655e6155dd54d529a3d007d1b3cffbe: Bootstrap starting.
I20260806 09:28:36.440514 24315 tablet_bootstrap.cc:492] T 47b56e8d01234329aaad38beacdf0fd4 P af3e642ec6e14f248dd046cf4cecafd3: Bootstrap starting.
I20260806 09:28:36.441154 24317 tablet_bootstrap.cc:654] T 47b56e8d01234329aaad38beacdf0fd4 P ff83a0fd6cb3436f8fe060c9ca900644: Neither blocks nor log segments found. Creating new log.
I20260806 09:28:36.442327 24315 tablet_bootstrap.cc:654] T 47b56e8d01234329aaad38beacdf0fd4 P af3e642ec6e14f248dd046cf4cecafd3: Neither blocks nor log segments found. Creating new log.
I20260806 09:28:36.444682 24312 tablet_bootstrap.cc:654] T 47b56e8d01234329aaad38beacdf0fd4 P 9655e6155dd54d529a3d007d1b3cffbe: Neither blocks nor log segments found. Creating new log.
I20260806 09:28:36.447160 24312 tablet_bootstrap.cc:492] T 47b56e8d01234329aaad38beacdf0fd4 P 9655e6155dd54d529a3d007d1b3cffbe: No bootstrap required, opened a new log
I20260806 09:28:36.447348 24312 ts_tablet_manager.cc:1397] T 47b56e8d01234329aaad38beacdf0fd4 P 9655e6155dd54d529a3d007d1b3cffbe: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.000s
I20260806 09:28:36.448366 24312 raft_consensus.cc:348] T 47b56e8d01234329aaad38beacdf0fd4 P 9655e6155dd54d529a3d007d1b3cffbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af3e642ec6e14f248dd046cf4cecafd3" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33057 } } peers { permanent_uuid: "9655e6155dd54d529a3d007d1b3cffbe" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 39021 } } peers { permanent_uuid: "ff83a0fd6cb3436f8fe060c9ca900644" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 34751 } }
I20260806 09:28:36.448697 24312 raft_consensus.cc:374] T 47b56e8d01234329aaad38beacdf0fd4 P 9655e6155dd54d529a3d007d1b3cffbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:28:36.448800 24312 raft_consensus.cc:729] T 47b56e8d01234329aaad38beacdf0fd4 P 9655e6155dd54d529a3d007d1b3cffbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9655e6155dd54d529a3d007d1b3cffbe, State: Initialized, Role: FOLLOWER
I20260806 09:28:36.449096 24312 consensus_queue.cc:260] T 47b56e8d01234329aaad38beacdf0fd4 P 9655e6155dd54d529a3d007d1b3cffbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af3e642ec6e14f248dd046cf4cecafd3" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33057 } } peers { permanent_uuid: "9655e6155dd54d529a3d007d1b3cffbe" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 39021 } } peers { permanent_uuid: "ff83a0fd6cb3436f8fe060c9ca900644" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 34751 } }
I20260806 09:28:36.449868 24312 ts_tablet_manager.cc:1428] T 47b56e8d01234329aaad38beacdf0fd4 P 9655e6155dd54d529a3d007d1b3cffbe: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
I20260806 09:28:36.453385 24317 tablet_bootstrap.cc:492] T 47b56e8d01234329aaad38beacdf0fd4 P ff83a0fd6cb3436f8fe060c9ca900644: No bootstrap required, opened a new log
I20260806 09:28:36.453577 24317 ts_tablet_manager.cc:1397] T 47b56e8d01234329aaad38beacdf0fd4 P ff83a0fd6cb3436f8fe060c9ca900644: Time spent bootstrapping tablet: real 0.014s user 0.007s sys 0.005s
I20260806 09:28:36.454572 24317 raft_consensus.cc:348] T 47b56e8d01234329aaad38beacdf0fd4 P ff83a0fd6cb3436f8fe060c9ca900644 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af3e642ec6e14f248dd046cf4cecafd3" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33057 } } peers { permanent_uuid: "9655e6155dd54d529a3d007d1b3cffbe" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 39021 } } peers { permanent_uuid: "ff83a0fd6cb3436f8fe060c9ca900644" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 34751 } }
I20260806 09:28:36.454928 24317 raft_consensus.cc:374] T 47b56e8d01234329aaad38beacdf0fd4 P ff83a0fd6cb3436f8fe060c9ca900644 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:28:36.455061 24317 raft_consensus.cc:729] T 47b56e8d01234329aaad38beacdf0fd4 P ff83a0fd6cb3436f8fe060c9ca900644 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff83a0fd6cb3436f8fe060c9ca900644, State: Initialized, Role: FOLLOWER
I20260806 09:28:36.455400 24317 consensus_queue.cc:260] T 47b56e8d01234329aaad38beacdf0fd4 P ff83a0fd6cb3436f8fe060c9ca900644 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af3e642ec6e14f248dd046cf4cecafd3" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33057 } } peers { permanent_uuid: "9655e6155dd54d529a3d007d1b3cffbe" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 39021 } } peers { permanent_uuid: "ff83a0fd6cb3436f8fe060c9ca900644" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 34751 } }
I20260806 09:28:36.456171 24317 ts_tablet_manager.cc:1428] T 47b56e8d01234329aaad38beacdf0fd4 P ff83a0fd6cb3436f8fe060c9ca900644: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260806 09:28:36.458699 24315 tablet_bootstrap.cc:492] T 47b56e8d01234329aaad38beacdf0fd4 P af3e642ec6e14f248dd046cf4cecafd3: No bootstrap required, opened a new log
I20260806 09:28:36.458897 24315 ts_tablet_manager.cc:1397] T 47b56e8d01234329aaad38beacdf0fd4 P af3e642ec6e14f248dd046cf4cecafd3: Time spent bootstrapping tablet: real 0.018s user 0.000s sys 0.010s
I20260806 09:28:36.459915 24315 raft_consensus.cc:348] T 47b56e8d01234329aaad38beacdf0fd4 P af3e642ec6e14f248dd046cf4cecafd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af3e642ec6e14f248dd046cf4cecafd3" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33057 } } peers { permanent_uuid: "9655e6155dd54d529a3d007d1b3cffbe" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 39021 } } peers { permanent_uuid: "ff83a0fd6cb3436f8fe060c9ca900644" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 34751 } }
I20260806 09:28:36.460284 24315 raft_consensus.cc:374] T 47b56e8d01234329aaad38beacdf0fd4 P af3e642ec6e14f248dd046cf4cecafd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:28:36.460400 24315 raft_consensus.cc:729] T 47b56e8d01234329aaad38beacdf0fd4 P af3e642ec6e14f248dd046cf4cecafd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af3e642ec6e14f248dd046cf4cecafd3, State: Initialized, Role: FOLLOWER
I20260806 09:28:36.460690 24315 consensus_queue.cc:260] T 47b56e8d01234329aaad38beacdf0fd4 P af3e642ec6e14f248dd046cf4cecafd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af3e642ec6e14f248dd046cf4cecafd3" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33057 } } peers { permanent_uuid: "9655e6155dd54d529a3d007d1b3cffbe" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 39021 } } peers { permanent_uuid: "ff83a0fd6cb3436f8fe060c9ca900644" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 34751 } }
I20260806 09:28:36.461236 24315 ts_tablet_manager.cc:1428] T 47b56e8d01234329aaad38beacdf0fd4 P af3e642ec6e14f248dd046cf4cecafd3: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260806 09:28:36.477789 24325 raft_consensus.cc:482] T 47b56e8d01234329aaad38beacdf0fd4 P 9655e6155dd54d529a3d007d1b3cffbe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 09:28:36.478001 24325 raft_consensus.cc:504] T 47b56e8d01234329aaad38beacdf0fd4 P 9655e6155dd54d529a3d007d1b3cffbe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af3e642ec6e14f248dd046cf4cecafd3" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33057 } } peers { permanent_uuid: "9655e6155dd54d529a3d007d1b3cffbe" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 39021 } } peers { permanent_uuid: "ff83a0fd6cb3436f8fe060c9ca900644" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 34751 } }
I20260806 09:28:36.479664 24325 leader_election.cc:288] T 47b56e8d01234329aaad38beacdf0fd4 P 9655e6155dd54d529a3d007d1b3cffbe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers af3e642ec6e14f248dd046cf4cecafd3 (127.20.113.1:33057), ff83a0fd6cb3436f8fe060c9ca900644 (127.20.113.3:34751)
I20260806 09:28:36.487227 24346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:36.479967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.1:33057 (local address 127.20.113.2:59661)
0806 09:28:36.481007 (+ 1040us) negotiation.cc:107] Waiting for socket to connect
0806 09:28:36.481069 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0806 09:28:36.481256 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:28:36.481622 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:28:36.481627 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:28:36.481655 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:28:36.482020 (+ 365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:36.482030 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:36.483216 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:36.483225 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:36.484962 (+ 1737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:36.484975 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:36.486576 (+ 1601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:36.486580 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:36.486691 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:36.486716 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 09:28:36.486799 (+ 83us) client_negotiation.cc:240] Negotiation successful
0806 09:28:36.486909 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":885,"thread_start_us":693,"threads_started":1}
I20260806 09:28:36.487308 24329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:36.480755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.2:59661 (local address 127.20.113.1:33057)
0806 09:28:36.480944 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:36.480949 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:36.481173 (+ 224us) server_negotiation.cc:407] Connection header received
0806 09:28:36.481430 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:36.481433 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:36.481493 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:36.481597 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:28:36.482208 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:36.482957 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:36.485204 (+ 2247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:36.486374 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:36.486540 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:36.486623 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:36.486850 (+ 227us) server_negotiation.cc:299] Negotiation successful
0806 09:28:36.486986 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260806 09:28:36.488250 23679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47b56e8d01234329aaad38beacdf0fd4" candidate_uuid: "9655e6155dd54d529a3d007d1b3cffbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af3e642ec6e14f248dd046cf4cecafd3" is_pre_election: true
I20260806 09:28:36.488584 23679 raft_consensus.cc:2454] T 47b56e8d01234329aaad38beacdf0fd4 P af3e642ec6e14f248dd046cf4cecafd3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9655e6155dd54d529a3d007d1b3cffbe in term 0.
I20260806 09:28:36.488657 24349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:36.480265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.3:34751 (local address 127.20.113.2:45109)
0806 09:28:36.481471 (+ 1206us) negotiation.cc:107] Waiting for socket to connect
0806 09:28:36.481484 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 09:28:36.481617 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:28:36.481990 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:28:36.481994 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:28:36.482024 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:28:36.482397 (+ 373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:36.482406 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:36.483718 (+ 1312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:36.483724 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:36.485647 (+ 1923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:36.485662 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:36.487911 (+ 2249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:36.487916 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:36.488041 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:36.488064 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 09:28:36.488166 (+ 102us) client_negotiation.cc:240] Negotiation successful
0806 09:28:36.488294 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1075,"thread_start_us":850,"threads_started":1}
I20260806 09:28:36.488690 24347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:36.480136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.2:45109 (local address 127.20.113.3:34751)
0806 09:28:36.480876 (+ 740us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:36.480883 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:36.481571 (+ 688us) server_negotiation.cc:407] Connection header received
0806 09:28:36.481774 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:36.481778 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:36.481858 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:36.481964 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:28:36.482565 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:36.483565 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:36.485844 (+ 2279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:36.487190 (+ 1346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:36.487500 (+ 310us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:36.487579 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:36.488210 (+ 631us) server_negotiation.cc:299] Negotiation successful
0806 09:28:36.488369 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":577,"thread_start_us":416,"threads_started":1}
I20260806 09:28:36.489113 23773 leader_election.cc:302] T 47b56e8d01234329aaad38beacdf0fd4 P 9655e6155dd54d529a3d007d1b3cffbe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9655e6155dd54d529a3d007d1b3cffbe, af3e642ec6e14f248dd046cf4cecafd3; no voters:
I20260806 09:28:36.489411 24078 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47b56e8d01234329aaad38beacdf0fd4" candidate_uuid: "9655e6155dd54d529a3d007d1b3cffbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff83a0fd6cb3436f8fe060c9ca900644" is_pre_election: true
I20260806 09:28:36.489550 24325 raft_consensus.cc:2781] T 47b56e8d01234329aaad38beacdf0fd4 P 9655e6155dd54d529a3d007d1b3cffbe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 09:28:36.489692 24325 raft_consensus.cc:482] T 47b56e8d01234329aaad38beacdf0fd4 P 9655e6155dd54d529a3d007d1b3cffbe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 09:28:36.489679 24078 raft_consensus.cc:2454] T 47b56e8d01234329aaad38beacdf0fd4 P ff83a0fd6cb3436f8fe060c9ca900644 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9655e6155dd54d529a3d007d1b3cffbe in term 0.
I20260806 09:28:36.489833 24325 raft_consensus.cc:3037] T 47b56e8d01234329aaad38beacdf0fd4 P 9655e6155dd54d529a3d007d1b3cffbe [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:28:36.492163 24325 raft_consensus.cc:504] T 47b56e8d01234329aaad38beacdf0fd4 P 9655e6155dd54d529a3d007d1b3cffbe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af3e642ec6e14f248dd046cf4cecafd3" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33057 } } peers { permanent_uuid: "9655e6155dd54d529a3d007d1b3cffbe" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 39021 } } peers { permanent_uuid: "ff83a0fd6cb3436f8fe060c9ca900644" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 34751 } }
I20260806 09:28:36.492825 24325 leader_election.cc:288] T 47b56e8d01234329aaad38beacdf0fd4 P 9655e6155dd54d529a3d007d1b3cffbe [CANDIDATE]: Term 1 election: Requested vote from peers af3e642ec6e14f248dd046cf4cecafd3 (127.20.113.1:33057), ff83a0fd6cb3436f8fe060c9ca900644 (127.20.113.3:34751)
I20260806 09:28:36.493268 23679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47b56e8d01234329aaad38beacdf0fd4" candidate_uuid: "9655e6155dd54d529a3d007d1b3cffbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af3e642ec6e14f248dd046cf4cecafd3"
I20260806 09:28:36.493323 24078 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47b56e8d01234329aaad38beacdf0fd4" candidate_uuid: "9655e6155dd54d529a3d007d1b3cffbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff83a0fd6cb3436f8fe060c9ca900644"
I20260806 09:28:36.493454 23679 raft_consensus.cc:3037] T 47b56e8d01234329aaad38beacdf0fd4 P af3e642ec6e14f248dd046cf4cecafd3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:28:36.493553 24078 raft_consensus.cc:3037] T 47b56e8d01234329aaad38beacdf0fd4 P ff83a0fd6cb3436f8fe060c9ca900644 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:28:36.495533 23679 raft_consensus.cc:2454] T 47b56e8d01234329aaad38beacdf0fd4 P af3e642ec6e14f248dd046cf4cecafd3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9655e6155dd54d529a3d007d1b3cffbe in term 1.
I20260806 09:28:36.495541 24078 raft_consensus.cc:2454] T 47b56e8d01234329aaad38beacdf0fd4 P ff83a0fd6cb3436f8fe060c9ca900644 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9655e6155dd54d529a3d007d1b3cffbe in term 1.
I20260806 09:28:36.495918 23775 leader_election.cc:302] T 47b56e8d01234329aaad38beacdf0fd4 P 9655e6155dd54d529a3d007d1b3cffbe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9655e6155dd54d529a3d007d1b3cffbe, ff83a0fd6cb3436f8fe060c9ca900644; no voters:
I20260806 09:28:36.496254 24325 raft_consensus.cc:2781] T 47b56e8d01234329aaad38beacdf0fd4 P 9655e6155dd54d529a3d007d1b3cffbe [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:28:36.496797 24325 raft_consensus.cc:686] T 47b56e8d01234329aaad38beacdf0fd4 P 9655e6155dd54d529a3d007d1b3cffbe [term 1 LEADER]: Becoming Leader. State: Replica: 9655e6155dd54d529a3d007d1b3cffbe, State: Running, Role: LEADER
I20260806 09:28:36.497326 24325 consensus_queue.cc:237] T 47b56e8d01234329aaad38beacdf0fd4 P 9655e6155dd54d529a3d007d1b3cffbe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af3e642ec6e14f248dd046cf4cecafd3" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33057 } } peers { permanent_uuid: "9655e6155dd54d529a3d007d1b3cffbe" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 39021 } } peers { permanent_uuid: "ff83a0fd6cb3436f8fe060c9ca900644" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 34751 } }
I20260806 09:28:36.500844 23447 catalog_manager.cc:5495] T 47b56e8d01234329aaad38beacdf0fd4 P 9655e6155dd54d529a3d007d1b3cffbe reported cstate change: term changed from 0 to 1, leader changed from <none> to 9655e6155dd54d529a3d007d1b3cffbe (127.20.113.2). New cstate: current_term: 1 leader_uuid: "9655e6155dd54d529a3d007d1b3cffbe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af3e642ec6e14f248dd046cf4cecafd3" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33057 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9655e6155dd54d529a3d007d1b3cffbe" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 39021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff83a0fd6cb3436f8fe060c9ca900644" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 34751 } health_report { overall_health: UNKNOWN } } }
I20260806 09:28:36.532670 24353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:36.526560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51358 (local address 127.20.113.2:39021)
0806 09:28:36.527250 (+ 690us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:36.527254 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:36.527273 (+ 19us) server_negotiation.cc:407] Connection header received
0806 09:28:36.527354 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:36.527357 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:36.527434 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:36.527586 (+ 152us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 09:28:36.528159 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:36.529033 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:36.530381 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:36.530805 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:36.530909 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:36.531356 (+ 447us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 09:28:36.531953 (+ 597us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 09:28:36.532061 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:36.532192 (+ 131us) server_negotiation.cc:299] Negotiation successful
0806 09:28:36.532292 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":576,"thread_start_us":452,"threads_started":1}
I20260806 09:28:36.548274 24329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:36.542238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.2:56679 (local address 127.20.113.1:33057)
0806 09:28:36.542481 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:36.542490 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:36.543253 (+ 763us) server_negotiation.cc:407] Connection header received
0806 09:28:36.543435 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:36.543439 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:36.543505 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:36.543579 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 09:28:36.544046 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:36.544908 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:36.546348 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:36.546864 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:36.546971 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:36.547440 (+ 469us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 09:28:36.547560 (+ 120us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 09:28:36.547626 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:36.547811 (+ 185us) server_negotiation.cc:299] Negotiation successful
0806 09:28:36.547896 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 09:28:36.548352 24357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:36.542234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.1:33057 (local address 127.20.113.2:56679)
0806 09:28:36.543105 (+ 871us) negotiation.cc:107] Waiting for socket to connect
0806 09:28:36.543122 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 09:28:36.543295 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:28:36.543621 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:28:36.543625 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:28:36.543634 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 09:28:36.543919 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:36.543927 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:36.545044 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:36.545048 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:36.546197 (+ 1149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:36.546210 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:36.547021 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:36.547044 (+ 23us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:36.547216 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:36.547284 (+ 68us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 09:28:36.547642 (+ 358us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 09:28:36.547651 (+ 9us) client_negotiation.cc:769] Sending connection context
0806 09:28:36.547735 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 09:28:36.547898 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":723,"thread_start_us":494,"threads_started":1}
I20260806 09:28:36.556869 23679 raft_consensus.cc:1261] T 47b56e8d01234329aaad38beacdf0fd4 P af3e642ec6e14f248dd046cf4cecafd3 [term 1 FOLLOWER]: Refusing update from remote peer 9655e6155dd54d529a3d007d1b3cffbe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 09:28:36.556905 24078 raft_consensus.cc:1261] T 47b56e8d01234329aaad38beacdf0fd4 P ff83a0fd6cb3436f8fe060c9ca900644 [term 1 FOLLOWER]: Refusing update from remote peer 9655e6155dd54d529a3d007d1b3cffbe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 09:28:36.557655 24350 consensus_queue.cc:1035] T 47b56e8d01234329aaad38beacdf0fd4 P 9655e6155dd54d529a3d007d1b3cffbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "af3e642ec6e14f248dd046cf4cecafd3" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33057 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 09:28:36.558046 24325 consensus_queue.cc:1035] T 47b56e8d01234329aaad38beacdf0fd4 P 9655e6155dd54d529a3d007d1b3cffbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff83a0fd6cb3436f8fe060c9ca900644" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 34751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 09:28:36.586531 24353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:36.580684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.1:48069 (local address 127.20.113.2:39021)
0806 09:28:36.580879 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:36.580885 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:36.581047 (+ 162us) server_negotiation.cc:407] Connection header received
0806 09:28:36.581242 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:36.581245 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:36.581307 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:36.581426 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 09:28:36.581932 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:36.582659 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:36.584497 (+ 1838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:36.585197 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:36.585335 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:36.585795 (+ 460us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 09:28:36.585892 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 09:28:36.585989 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:36.586111 (+ 122us) server_negotiation.cc:299] Negotiation successful
0806 09:28:36.586186 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260806 09:28:36.586647 24362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:36.579946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.2:39021 (local address 127.20.113.1:48069)
0806 09:28:36.580939 (+ 993us) negotiation.cc:107] Waiting for socket to connect
0806 09:28:36.580955 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 09:28:36.581111 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:28:36.581456 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:28:36.581460 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:28:36.581470 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 09:28:36.581785 (+ 315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:36.581793 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:36.582837 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:36.582841 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:36.584361 (+ 1520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:36.584375 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:36.585398 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:36.585404 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:36.585560 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:36.585597 (+ 37us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 09:28:36.585994 (+ 397us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 09:28:36.586008 (+ 14us) client_negotiation.cc:769] Sending connection context
0806 09:28:36.586084 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 09:28:36.586211 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":850,"thread_start_us":670,"threads_started":1}
W20260806 09:28:36.587713 24272 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260806 09:28:38.208106 23447 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38130:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 09:28:38.208455 23447 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38130:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 09:28:38.212860 23447 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 405697530ec54e9eac6662969bfb3711: Sending DeleteTablet for 3 replicas of tablet 47b56e8d01234329aaad38beacdf0fd4
I20260806 09:28:38.213761 23840 tablet_service.cc:1514] Processing DeleteTablet for tablet 47b56e8d01234329aaad38beacdf0fd4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:28:38 UTC) from {username='kudu', principal='kudu/127.20.113.62@KRBTEST.COM'} at 127.0.0.1:51346
I20260806 09:28:38.213796 23641 tablet_service.cc:1514] Processing DeleteTablet for tablet 47b56e8d01234329aaad38beacdf0fd4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:28:38 UTC) from {username='kudu', principal='kudu/127.20.113.62@KRBTEST.COM'} at 127.0.0.1:41522
I20260806 09:28:38.214054 24042 tablet_service.cc:1514] Processing DeleteTablet for tablet 47b56e8d01234329aaad38beacdf0fd4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:28:38 UTC) from {username='kudu', principal='kudu/127.20.113.62@KRBTEST.COM'} at 127.0.0.1:49678
I20260806 09:28:38.215106 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 23551
I20260806 09:28:38.227329 24398 tablet_replica.cc:330] T 47b56e8d01234329aaad38beacdf0fd4 P 9655e6155dd54d529a3d007d1b3cffbe: stopping tablet replica
I20260806 09:28:38.227559 24398 raft_consensus.cc:2229] T 47b56e8d01234329aaad38beacdf0fd4 P 9655e6155dd54d529a3d007d1b3cffbe [term 1 LEADER]: Raft consensus shutting down.
I20260806 09:28:38.228124 24398 raft_consensus.cc:2258] T 47b56e8d01234329aaad38beacdf0fd4 P 9655e6155dd54d529a3d007d1b3cffbe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 09:28:38.228834 24399 tablet_replica.cc:330] T 47b56e8d01234329aaad38beacdf0fd4 P af3e642ec6e14f248dd046cf4cecafd3: stopping tablet replica
I20260806 09:28:38.229115 24399 raft_consensus.cc:2229] T 47b56e8d01234329aaad38beacdf0fd4 P af3e642ec6e14f248dd046cf4cecafd3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 09:28:38.229401 24399 raft_consensus.cc:2258] T 47b56e8d01234329aaad38beacdf0fd4 P af3e642ec6e14f248dd046cf4cecafd3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 09:28:38.230221 24400 tablet_replica.cc:330] T 47b56e8d01234329aaad38beacdf0fd4 P ff83a0fd6cb3436f8fe060c9ca900644: stopping tablet replica
I20260806 09:28:38.230433 24400 raft_consensus.cc:2229] T 47b56e8d01234329aaad38beacdf0fd4 P ff83a0fd6cb3436f8fe060c9ca900644 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 09:28:38.230523 24399 ts_tablet_manager.cc:1905] T 47b56e8d01234329aaad38beacdf0fd4 P af3e642ec6e14f248dd046cf4cecafd3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 09:28:38.230708 24400 raft_consensus.cc:2258] T 47b56e8d01234329aaad38beacdf0fd4 P ff83a0fd6cb3436f8fe060c9ca900644 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 09:28:38.229473 24398 ts_tablet_manager.cc:1905] T 47b56e8d01234329aaad38beacdf0fd4 P 9655e6155dd54d529a3d007d1b3cffbe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 09:28:38.233292 24400 ts_tablet_manager.cc:1905] T 47b56e8d01234329aaad38beacdf0fd4 P ff83a0fd6cb3436f8fe060c9ca900644: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 09:28:38.237957 24398 ts_tablet_manager.cc:1918] T 47b56e8d01234329aaad38beacdf0fd4 P 9655e6155dd54d529a3d007d1b3cffbe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 09:28:38.238092 24398 log.cc:1196] T 47b56e8d01234329aaad38beacdf0fd4 P 9655e6155dd54d529a3d007d1b3cffbe: Deleting WAL directory at /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786008495763054-20932-0/minicluster-data/ts-1/wal/wals/47b56e8d01234329aaad38beacdf0fd4
I20260806 09:28:38.238526 24399 ts_tablet_manager.cc:1918] T 47b56e8d01234329aaad38beacdf0fd4 P af3e642ec6e14f248dd046cf4cecafd3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 09:28:38.238585 24398 ts_tablet_manager.cc:1939] T 47b56e8d01234329aaad38beacdf0fd4 P 9655e6155dd54d529a3d007d1b3cffbe: Deleting consensus metadata
I20260806 09:28:38.238668 24399 log.cc:1196] T 47b56e8d01234329aaad38beacdf0fd4 P af3e642ec6e14f248dd046cf4cecafd3: Deleting WAL directory at /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786008495763054-20932-0/minicluster-data/ts-0/wal/wals/47b56e8d01234329aaad38beacdf0fd4
I20260806 09:28:38.239233 24399 ts_tablet_manager.cc:1939] T 47b56e8d01234329aaad38beacdf0fd4 P af3e642ec6e14f248dd046cf4cecafd3: Deleting consensus metadata
I20260806 09:28:38.239718 24400 ts_tablet_manager.cc:1918] T 47b56e8d01234329aaad38beacdf0fd4 P ff83a0fd6cb3436f8fe060c9ca900644: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 09:28:38.239902 24400 log.cc:1196] T 47b56e8d01234329aaad38beacdf0fd4 P ff83a0fd6cb3436f8fe060c9ca900644: Deleting WAL directory at /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786008495763054-20932-0/minicluster-data/ts-2/wal/wals/47b56e8d01234329aaad38beacdf0fd4
I20260806 09:28:38.239907 23433 catalog_manager.cc:4841] TS 9655e6155dd54d529a3d007d1b3cffbe (127.20.113.2:39021): tablet 47b56e8d01234329aaad38beacdf0fd4 (table test-table [id=8083197781c8495abe3c97e50b9c28e0]) successfully deleted
I20260806 09:28:38.240449 24400 ts_tablet_manager.cc:1939] T 47b56e8d01234329aaad38beacdf0fd4 P ff83a0fd6cb3436f8fe060c9ca900644: Deleting consensus metadata
I20260806 09:28:38.240458 23431 catalog_manager.cc:4841] TS af3e642ec6e14f248dd046cf4cecafd3 (127.20.113.1:33057): tablet 47b56e8d01234329aaad38beacdf0fd4 (table test-table [id=8083197781c8495abe3c97e50b9c28e0]) successfully deleted
I20260806 09:28:38.241609 23432 catalog_manager.cc:4841] TS ff83a0fd6cb3436f8fe060c9ca900644 (127.20.113.3:34751): tablet 47b56e8d01234329aaad38beacdf0fd4 (table test-table [id=8083197781c8495abe3c97e50b9c28e0]) successfully deleted
Aug 06 09:28:38 dist-test-slave-45dx krb5kdc[23391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008515, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.20.113.1@KRBTEST.COM
I20260806 09:28:38.250588 24402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:38.218184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41564 (local address 127.20.113.1:33057)
0806 09:28:38.229249 (+ 11065us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:38.229258 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:38.229285 (+ 27us) server_negotiation.cc:407] Connection header received
0806 09:28:38.229346 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:38.229351 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:38.229440 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:38.229578 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:28:38.231447 (+ 1869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:38.232505 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:38.234449 (+ 1944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:38.235200 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:38.235358 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:38.244399 (+ 9041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:28:38.244417 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:28:38.244435 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:28:38.244473 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:28:38.247910 (+ 3437us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:38.248739 (+ 829us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:38.248754 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:38.248757 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:38.248912 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:38.249289 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:38.249292 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:38.249294 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:38.249649 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:28:38.249914 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:38.250135 (+ 221us) server_negotiation.cc:299] Negotiation successful
0806 09:28:38.250321 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10918,"thread_start_us":11497,"threads_started":1}
W20260806 09:28:38.251353 23731 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41564
W20260806 09:28:38.252005 20932 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 23551: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 09:28:38.252241 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 23551
I20260806 09:28:38.296135 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 23744
Aug 06 09:28:38 dist-test-slave-45dx krb5kdc[23391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008515, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.20.113.2@KRBTEST.COM
I20260806 09:28:38.312342 24403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:38.297014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51362 (local address 127.20.113.2:39021)
0806 09:28:38.297533 (+ 519us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:38.297537 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:38.297552 (+ 15us) server_negotiation.cc:407] Connection header received
0806 09:28:38.297605 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:38.297608 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:38.297670 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:38.297791 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:28:38.299053 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:38.299803 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:38.301263 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:38.301736 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:38.301922 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:38.306288 (+ 4366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:28:38.306305 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:28:38.306322 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:28:38.306362 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:28:38.309241 (+ 2879us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:38.309895 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:38.309903 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:38.309905 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:38.310131 (+ 226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:38.310553 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:38.310558 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:38.310562 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:38.310952 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:28:38.311290 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:38.311606 (+ 316us) server_negotiation.cc:299] Negotiation successful
0806 09:28:38.311888 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":414,"thread_start_us":262,"threads_started":1}
W20260806 09:28:38.313051 23938 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51362
W20260806 09:28:38.313740 20932 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 23744: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 09:28:38.313936 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 23744
I20260806 09:28:38.348716 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 23950
Aug 06 09:28:38 dist-test-slave-45dx krb5kdc[23391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008515, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.20.113.3@KRBTEST.COM
I20260806 09:28:38.363938 24406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:38.349533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49694 (local address 127.20.113.3:34751)
0806 09:28:38.350094 (+ 561us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:38.350098 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:38.350115 (+ 17us) server_negotiation.cc:407] Connection header received
0806 09:28:38.350172 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:38.350176 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:38.350236 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:38.350351 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:28:38.351662 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:38.352442 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:38.353629 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:38.354007 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:38.354111 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:38.358410 (+ 4299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:28:38.358427 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:28:38.358444 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:28:38.358479 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:28:38.361501 (+ 3022us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:38.362138 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:38.362146 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:38.362148 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:38.362297 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:38.362621 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:38.362624 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:38.362626 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:38.362985 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:28:38.363269 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:38.363456 (+ 187us) server_negotiation.cc:299] Negotiation successful
0806 09:28:38.363650 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":439,"thread_start_us":303,"threads_started":1}
W20260806 09:28:38.364730 24144 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49694
W20260806 09:28:38.365381 20932 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 23950: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 09:28:38.365577 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 23950
I20260806 09:28:38.406545 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 23411
I20260806 09:28:38.407869 23510 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:28:38.528188 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 23411
2026-08-06T09:28:38Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (8821 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786008495763054-20932-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 09:28:38 dist-test-slave-45dx krb5kdc[24411](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 09:28:38 dist-test-slave-45dx krb5kdc[24411](info): set up 2 sockets
Aug 06 09:28:38 dist-test-slave-45dx krb5kdc[24411](info): commencing operation
krb5kdc: starting...
W20260806 09:28:39.910012 20932 mini_kdc.cc:121] Time spent starting KDC: real 1.328s user 0.004s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
W20260806 09:28:40.037595 20932 mini_kdc.cc:253] Time spent creating user principal test-admin: real 0.127s user 0.000s sys 0.009s
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
W20260806 09:28:40.224136 20932 mini_kdc.cc:253] Time spent creating user principal test-user: real 0.186s user 0.000s sys 0.009s
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
W20260806 09:28:40.357120 20932 mini_kdc.cc:253] Time spent creating user principal joe-interloper: real 0.133s user 0.003s sys 0.006s
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786008495763054-20932-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786008495763054-20932-0/krb5kdc/test-user.keytab.
W20260806 09:28:40.804131 20932 mini_kdc.cc:302] Time spent creating keytab for test-user: real 0.447s user 0.000s sys 0.009s
Aug 06 09:28:40 dist-test-slave-45dx krb5kdc[24411](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008520, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T09:28:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T09:28:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.113.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.113.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.113.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab.
Entry for principal kudu/127.20.113.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab.
I20260806 09:28:40.910625 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786008495763054-20932-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786008495763054-20932-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786008495763054-20932-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.113.62:36461
--webserver_interface=127.20.113.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab
--principal=kudu/127.20.113.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:45167
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.113.62:36461
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:28:41.241252 24433 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:28:41.241535 24433 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:28:41.241708 24433 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:28:41.248831 24433 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 09:28:41.248962 24433 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:28:41.249073 24433 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:28:41.249184 24433 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 09:28:41.249269 24433 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 09:28:41.258810 24433 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:45167
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786008495763054-20932-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786008495763054-20932-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.113.62:36461
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab
--principal=kudu/127.20.113.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.113.62:36461
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.113.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786008495763054-20932-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:28:41.260195 24433 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:28:41.261656 24433 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:28:41.273705 24439 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:28:41.273661 24438 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:28:41.273728 24446 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:28:41.274391 24433 server_base.cc:1034] running on GCE node
I20260806 09:28:41.275182 24433 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:28:41.276515 24433 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:28:41.277711 24433 hybrid_clock.cc:648] HybridClock initialized: now 1786008521277676 us; error 33 us; skew 500 ppm
I20260806 09:28:41.278069 24433 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:28:41 dist-test-slave-45dx krb5kdc[24411](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008521, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:28:41.281347 24433 init.cc:377] Logged in from keytab as kudu/127.20.113.62@KRBTEST.COM (short username kudu)
I20260806 09:28:41.282781 24433 webserver.cc:460] Webserver started at http://127.20.113.62:37281/ using document root <none> and password file <none>
I20260806 09:28:41.283468 24433 fs_manager.cc:359] Metadata directory not provided
I20260806 09:28:41.283686 24433 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:28:41.284094 24433 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:28:41.287155 24433 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786008495763054-20932-0/minicluster-data/master-0/data/instance:
uuid: "ce601b2eb3f74e64998828a677cd3983"
format_stamp: "Formatted at 2026-08-06 09:28:41 on dist-test-slave-45dx"
server_key: "c2a5cec58758fdb5483df8ed2fe3ad98"
server_key_iv: "6649184b9d12ed063941ae549cd0cba2"
server_key_version: "encryptionkey@0"
I20260806 09:28:41.288003 24433 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786008495763054-20932-0/minicluster-data/master-0/wal/instance:
uuid: "ce601b2eb3f74e64998828a677cd3983"
format_stamp: "Formatted at 2026-08-06 09:28:41 on dist-test-slave-45dx"
server_key: "c2a5cec58758fdb5483df8ed2fe3ad98"
server_key_iv: "6649184b9d12ed063941ae549cd0cba2"
server_key_version: "encryptionkey@0"
I20260806 09:28:41.293467 24433 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260806 09:28:41.297012 24458 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:28:41.298298 24433 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 09:28:41.298472 24433 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786008495763054-20932-0/minicluster-data/master-0/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786008495763054-20932-0/minicluster-data/master-0/wal
uuid: "ce601b2eb3f74e64998828a677cd3983"
format_stamp: "Formatted at 2026-08-06 09:28:41 on dist-test-slave-45dx"
server_key: "c2a5cec58758fdb5483df8ed2fe3ad98"
server_key_iv: "6649184b9d12ed063941ae549cd0cba2"
server_key_version: "encryptionkey@0"
I20260806 09:28:41.298658 24433 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786008495763054-20932-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786008495763054-20932-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786008495763054-20932-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:28:41.314090 24433 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:28:41.318284 24433 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:28:41.318562 24433 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:28:41.338277 24433 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.62:36461
I20260806 09:28:41.338302 24535 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.62:36461 every 8 connection(s)
I20260806 09:28:41.339756 24433 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
I20260806 09:28:41.340914 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 24433
I20260806 09:28:41.341178 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786008495763054-20932-0/minicluster-data/master-0/wal/instance
I20260806 09:28:41.342099 20932 external_mini_cluster.cc:1428] Setting key e88fe4efad72d79f6217d2c705c987b2
I20260806 09:28:41.344889 24537 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 06 09:28:41 dist-test-slave-45dx krb5kdc[24411](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008520, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.113.62@KRBTEST.COM
I20260806 09:28:41.358031 24537 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce601b2eb3f74e64998828a677cd3983: Bootstrap starting.
I20260806 09:28:41.361269 24537 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ce601b2eb3f74e64998828a677cd3983: Neither blocks nor log segments found. Creating new log.
I20260806 09:28:41.362303 24537 log.cc:824] T 00000000000000000000000000000000 P ce601b2eb3f74e64998828a677cd3983: Log is configured to *not* fsync() on all Append() calls
I20260806 09:28:41.365185 24542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:41.344177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41684 (local address 127.20.113.62:36461)
0806 09:28:41.345111 (+ 934us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:41.345122 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:41.345166 (+ 44us) server_negotiation.cc:407] Connection header received
0806 09:28:41.346786 (+ 1620us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:41.346805 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:41.347160 (+ 355us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:41.347466 (+ 306us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:28:41.349286 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:41.350643 (+ 1357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:41.352445 (+ 1802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:41.353079 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:41.353293 (+ 214us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:41.357831 (+ 4538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:28:41.357838 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:28:41.357858 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:28:41.357893 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:28:41.361091 (+ 3198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:41.361885 (+ 794us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:41.361891 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:41.361898 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:41.362073 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:41.362471 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:41.362483 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:41.362485 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:41.363211 (+ 726us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:28:41.363532 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:41.363827 (+ 295us) server_negotiation.cc:299] Negotiation successful
0806 09:28:41.364172 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":690,"thread_start_us":443,"threads_started":1}
I20260806 09:28:41.365535 24537 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce601b2eb3f74e64998828a677cd3983: No bootstrap required, opened a new log
I20260806 09:28:41.373093 24537 raft_consensus.cc:348] T 00000000000000000000000000000000 P ce601b2eb3f74e64998828a677cd3983 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce601b2eb3f74e64998828a677cd3983" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 36461 } }
I20260806 09:28:41.373427 24537 raft_consensus.cc:374] T 00000000000000000000000000000000 P ce601b2eb3f74e64998828a677cd3983 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:28:41.373540 24537 raft_consensus.cc:729] T 00000000000000000000000000000000 P ce601b2eb3f74e64998828a677cd3983 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce601b2eb3f74e64998828a677cd3983, State: Initialized, Role: FOLLOWER
I20260806 09:28:41.374162 24537 consensus_queue.cc:260] T 00000000000000000000000000000000 P ce601b2eb3f74e64998828a677cd3983 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce601b2eb3f74e64998828a677cd3983" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 36461 } }
I20260806 09:28:41.374433 24537 raft_consensus.cc:388] T 00000000000000000000000000000000 P ce601b2eb3f74e64998828a677cd3983 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 09:28:41.374571 24537 raft_consensus.cc:482] T 00000000000000000000000000000000 P ce601b2eb3f74e64998828a677cd3983 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 09:28:41.374747 24537 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ce601b2eb3f74e64998828a677cd3983 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:28:41.376840 24537 raft_consensus.cc:504] T 00000000000000000000000000000000 P ce601b2eb3f74e64998828a677cd3983 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce601b2eb3f74e64998828a677cd3983" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 36461 } }
I20260806 09:28:41.377315 24537 leader_election.cc:302] T 00000000000000000000000000000000 P ce601b2eb3f74e64998828a677cd3983 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ce601b2eb3f74e64998828a677cd3983; no voters:
I20260806 09:28:41.377873 24537 leader_election.cc:288] T 00000000000000000000000000000000 P ce601b2eb3f74e64998828a677cd3983 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 09:28:41.378005 24544 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ce601b2eb3f74e64998828a677cd3983 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:28:41.378888 24544 raft_consensus.cc:686] T 00000000000000000000000000000000 P ce601b2eb3f74e64998828a677cd3983 [term 1 LEADER]: Becoming Leader. State: Replica: ce601b2eb3f74e64998828a677cd3983, State: Running, Role: LEADER
I20260806 09:28:41.379331 24544 consensus_queue.cc:237] T 00000000000000000000000000000000 P ce601b2eb3f74e64998828a677cd3983 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce601b2eb3f74e64998828a677cd3983" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 36461 } }
I20260806 09:28:41.380111 24537 sys_catalog.cc:564] T 00000000000000000000000000000000 P ce601b2eb3f74e64998828a677cd3983 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 09:28:41.382445 24546 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce601b2eb3f74e64998828a677cd3983 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ce601b2eb3f74e64998828a677cd3983. Latest consensus state: current_term: 1 leader_uuid: "ce601b2eb3f74e64998828a677cd3983" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce601b2eb3f74e64998828a677cd3983" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 36461 } } }
I20260806 09:28:41.382913 24546 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce601b2eb3f74e64998828a677cd3983 [sys.catalog]: This master's current role is: LEADER
I20260806 09:28:41.383486 24545 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce601b2eb3f74e64998828a677cd3983 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ce601b2eb3f74e64998828a677cd3983" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce601b2eb3f74e64998828a677cd3983" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 36461 } } }
I20260806 09:28:41.383854 24545 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce601b2eb3f74e64998828a677cd3983 [sys.catalog]: This master's current role is: LEADER
I20260806 09:28:41.387125 24556 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 09:28:41.392227 24556 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 09:28:41.400827 24556 catalog_manager.cc:1347] Generated new cluster ID: 5e818309c62042acbbcb7a4b329de170
I20260806 09:28:41.400943 24556 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 09:28:41.408701 24556 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 09:28:41.409855 24556 catalog_manager.cc:1504] Loading token signing keys...
I20260806 09:28:41.418190 24556 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ce601b2eb3f74e64998828a677cd3983: Generated new TSK 0
I20260806 09:28:41.419009 24556 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.113.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.113.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.113.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.1.keytab.
Entry for principal kudu/127.20.113.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.1.keytab.
I20260806 09:28:41.488022 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786008495763054-20932-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786008495763054-20932-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786008495763054-20932-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786008495763054-20932-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.113.1:0
--local_ip_for_outbound_sockets=127.20.113.1
--webserver_interface=127.20.113.1
--webserver_port=0
--tserver_master_addrs=127.20.113.62:36461
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.1.keytab
--principal=kudu/127.20.113.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:45167
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:28:41.831544 24577 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:28:41.831902 24577 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:28:41.832093 24577 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:28:41.839314 24577 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:28:41.839458 24577 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:28:41.839643 24577 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.113.1
I20260806 09:28:41.849208 24577 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:45167
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786008495763054-20932-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786008495763054-20932-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.1.keytab
--principal=kudu/127.20.113.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.113.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786008495763054-20932-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.113.1
--webserver_port=0
--tserver_master_addrs=127.20.113.62:36461
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.113.1
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786008495763054-20932-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:28:41.850493 24577 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:28:41.851789 24577 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:28:41.865167 24585 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:28:41.865834 24587 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:28:41.865919 24589 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:28:41.866325 24577 server_base.cc:1034] running on GCE node
I20260806 09:28:41.866928 24577 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:28:41.867867 24577 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:28:41.869091 24577 hybrid_clock.cc:648] HybridClock initialized: now 1786008521869051 us; error 51 us; skew 500 ppm
I20260806 09:28:41.869398 24577 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:28:41 dist-test-slave-45dx krb5kdc[24411](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008521, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:28:41.872597 24577 init.cc:377] Logged in from keytab as kudu/127.20.113.1@KRBTEST.COM (short username kudu)
I20260806 09:28:41.874042 24577 webserver.cc:460] Webserver started at http://127.20.113.1:40761/ using document root <none> and password file <none>
I20260806 09:28:41.874701 24577 fs_manager.cc:359] Metadata directory not provided
I20260806 09:28:41.874843 24577 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:28:41.875151 24577 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:28:41.878005 24577 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786008495763054-20932-0/minicluster-data/ts-0/data/instance:
uuid: "8bbfa638495b4ec4b018fa5d206bd791"
format_stamp: "Formatted at 2026-08-06 09:28:41 on dist-test-slave-45dx"
server_key: "e0a43dfc2243abd60957b2b9c5048e89"
server_key_iv: "e8f9eb963342e9b9041164f93a8c471a"
server_key_version: "encryptionkey@0"
I20260806 09:28:41.878782 24577 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786008495763054-20932-0/minicluster-data/ts-0/wal/instance:
uuid: "8bbfa638495b4ec4b018fa5d206bd791"
format_stamp: "Formatted at 2026-08-06 09:28:41 on dist-test-slave-45dx"
server_key: "e0a43dfc2243abd60957b2b9c5048e89"
server_key_iv: "e8f9eb963342e9b9041164f93a8c471a"
server_key_version: "encryptionkey@0"
I20260806 09:28:41.883891 24577 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 09:28:41.887463 24600 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:28:41.888911 24577 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260806 09:28:41.889042 24577 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786008495763054-20932-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786008495763054-20932-0/minicluster-data/ts-0/wal
uuid: "8bbfa638495b4ec4b018fa5d206bd791"
format_stamp: "Formatted at 2026-08-06 09:28:41 on dist-test-slave-45dx"
server_key: "e0a43dfc2243abd60957b2b9c5048e89"
server_key_iv: "e8f9eb963342e9b9041164f93a8c471a"
server_key_version: "encryptionkey@0"
I20260806 09:28:41.889199 24577 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786008495763054-20932-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786008495763054-20932-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786008495763054-20932-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:28:41.919356 24577 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:28:41.923452 24577 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:28:41.923702 24577 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:28:41.924646 24577 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:28:41.925951 24577 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:28:41.926025 24577 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:28:41.926117 24577 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:28:41.926170 24577 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:28:41.960659 24577 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.1:37577
I20260806 09:28:41.960711 24751 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.1:37577 every 8 connection(s)
I20260806 09:28:41.962036 24577 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786008495763054-20932-0/minicluster-data/ts-0/data/info.pb
I20260806 09:28:41.963967 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 24577
I20260806 09:28:41.964222 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786008495763054-20932-0/minicluster-data/ts-0/wal/instance
I20260806 09:28:41.965020 20932 external_mini_cluster.cc:1428] Setting key ca8e17d6086981fc237d9893ef2ea4a3
Aug 06 09:28:41 dist-test-slave-45dx krb5kdc[24411](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008521, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.1@KRBTEST.COM for kudu/127.20.113.62@KRBTEST.COM
I20260806 09:28:41.984151 24754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:41.965219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.1:48903 (local address 127.20.113.62:36461)
0806 09:28:41.965725 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:41.965730 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:41.967179 (+ 1449us) server_negotiation.cc:407] Connection header received
0806 09:28:41.968881 (+ 1702us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:41.968890 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:41.968996 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:41.969133 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:28:41.970597 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:41.971708 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:41.973339 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:41.974069 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:41.974213 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:41.978166 (+ 3953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:28:41.978171 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:28:41.978174 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:28:41.978208 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:28:41.981011 (+ 2803us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:41.981789 (+ 778us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:41.981797 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:41.981805 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:41.981930 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:41.982392 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:41.982396 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:41.982398 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:41.982663 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:28:41.982823 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:41.983536 (+ 713us) server_negotiation.cc:299] Negotiation successful
0806 09:28:41.983787 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":361,"thread_start_us":228,"threads_started":1}
I20260806 09:28:41.985141 24757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:41.965809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:36461 (local address 127.20.113.1:48903)
0806 09:28:41.966720 (+ 911us) negotiation.cc:107] Waiting for socket to connect
0806 09:28:41.966765 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 09:28:41.968615 (+ 1850us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:28:41.969331 (+ 716us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:28:41.969338 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:28:41.969840 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:28:41.970417 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:41.970435 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:41.971868 (+ 1433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:41.971872 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:41.973194 (+ 1322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:41.973206 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:41.974267 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:41.974272 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:41.974470 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:41.975162 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:28:41.975184 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:28:41.977918 (+ 2734us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:28:41.981149 (+ 3231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:28:41.981158 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:28:41.981173 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:28:41.981647 (+ 474us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:28:41.982043 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:28:41.982047 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:28:41.982051 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:28:41.982247 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:28:41.982798 (+ 551us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:28:41.982805 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:28:41.983142 (+ 337us) client_negotiation.cc:769] Sending connection context
0806 09:28:41.983392 (+ 250us) client_negotiation.cc:240] Negotiation successful
0806 09:28:41.983971 (+ 579us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":621,"thread_start_us":387,"threads_started":1}
I20260806 09:28:41.986824 24752 heartbeater.cc:347] Connected to a master server at 127.20.113.62:36461
I20260806 09:28:41.987116 24752 heartbeater.cc:464] Registering TS with master...
I20260806 09:28:41.987926 24752 heartbeater.cc:511] Master 127.20.113.62:36461 requested a full tablet report, sending...
I20260806 09:28:41.990370 24483 ts_manager.cc:194] Registered new tserver with Master: 8bbfa638495b4ec4b018fa5d206bd791 (127.20.113.1:37577)
WARNING: no policy specified for kudu/127.20.113.2@KRBTEST.COM; defaulting to no policy
I20260806 09:28:41.992178 24483 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.113.1@KRBTEST.COM'} at 127.20.113.1:48903
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.113.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.113.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.2.keytab.
Entry for principal kudu/127.20.113.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.2.keytab.
I20260806 09:28:42.020821 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786008495763054-20932-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786008495763054-20932-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786008495763054-20932-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786008495763054-20932-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.113.2:0
--local_ip_for_outbound_sockets=127.20.113.2
--webserver_interface=127.20.113.2
--webserver_port=0
--tserver_master_addrs=127.20.113.62:36461
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.2.keytab
--principal=kudu/127.20.113.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:45167
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:28:42.358759 24764 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:28:42.359061 24764 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:28:42.359175 24764 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:28:42.366513 24764 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:28:42.366616 24764 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:28:42.366765 24764 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.113.2
I20260806 09:28:42.375993 24764 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:45167
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786008495763054-20932-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786008495763054-20932-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.2.keytab
--principal=kudu/127.20.113.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.113.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786008495763054-20932-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.113.2
--webserver_port=0
--tserver_master_addrs=127.20.113.62:36461
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.113.2
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786008495763054-20932-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:28:42.377343 24764 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:28:42.378718 24764 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:28:42.392086 24771 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:28:42.392134 24774 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:28:42.392439 24772 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:28:42.393452 24764 server_base.cc:1034] running on GCE node
I20260806 09:28:42.394122 24764 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:28:42.395056 24764 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:28:42.396344 24764 hybrid_clock.cc:648] HybridClock initialized: now 1786008522396245 us; error 57 us; skew 500 ppm
I20260806 09:28:42.396715 24764 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:28:42 dist-test-slave-45dx krb5kdc[24411](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008522, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:28:42.399757 24764 init.cc:377] Logged in from keytab as kudu/127.20.113.2@KRBTEST.COM (short username kudu)
I20260806 09:28:42.401358 24764 webserver.cc:460] Webserver started at http://127.20.113.2:35615/ using document root <none> and password file <none>
I20260806 09:28:42.402110 24764 fs_manager.cc:359] Metadata directory not provided
I20260806 09:28:42.402261 24764 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:28:42.402585 24764 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:28:42.405551 24764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786008495763054-20932-0/minicluster-data/ts-1/data/instance:
uuid: "a5dd484f3d4f49f394a15251febb8a42"
format_stamp: "Formatted at 2026-08-06 09:28:42 on dist-test-slave-45dx"
server_key: "c9ce612fa82d2fdde691b4ad3c146dd9"
server_key_iv: "4398bde0fedda00aec790aa83ae582fc"
server_key_version: "encryptionkey@0"
I20260806 09:28:42.406404 24764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786008495763054-20932-0/minicluster-data/ts-1/wal/instance:
uuid: "a5dd484f3d4f49f394a15251febb8a42"
format_stamp: "Formatted at 2026-08-06 09:28:42 on dist-test-slave-45dx"
server_key: "c9ce612fa82d2fdde691b4ad3c146dd9"
server_key_iv: "4398bde0fedda00aec790aa83ae582fc"
server_key_version: "encryptionkey@0"
I20260806 09:28:42.411504 24764 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260806 09:28:42.414775 24787 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:28:42.416015 24764 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 09:28:42.416193 24764 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786008495763054-20932-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786008495763054-20932-0/minicluster-data/ts-1/wal
uuid: "a5dd484f3d4f49f394a15251febb8a42"
format_stamp: "Formatted at 2026-08-06 09:28:42 on dist-test-slave-45dx"
server_key: "c9ce612fa82d2fdde691b4ad3c146dd9"
server_key_iv: "4398bde0fedda00aec790aa83ae582fc"
server_key_version: "encryptionkey@0"
I20260806 09:28:42.416440 24764 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786008495763054-20932-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786008495763054-20932-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786008495763054-20932-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:28:42.433777 24764 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:28:42.437875 24764 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:28:42.438182 24764 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:28:42.439186 24764 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:28:42.440609 24764 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:28:42.440717 24764 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:28:42.440878 24764 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:28:42.440955 24764 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:28:42.476138 24764 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.2:36711
I20260806 09:28:42.476184 24959 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.2:36711 every 8 connection(s)
I20260806 09:28:42.477479 24764 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786008495763054-20932-0/minicluster-data/ts-1/data/info.pb
I20260806 09:28:42.482939 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 24764
I20260806 09:28:42.483150 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786008495763054-20932-0/minicluster-data/ts-1/wal/instance
I20260806 09:28:42.483714 20932 external_mini_cluster.cc:1428] Setting key e3e44b05820705f7ccbb9e87163e47f3
Aug 06 09:28:42 dist-test-slave-45dx krb5kdc[24411](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008522, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.2@KRBTEST.COM for kudu/127.20.113.62@KRBTEST.COM
I20260806 09:28:42.500806 24754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:42.480073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.2:60697 (local address 127.20.113.62:36461)
0806 09:28:42.480296 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:42.480343 (+ 47us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:42.481774 (+ 1431us) server_negotiation.cc:407] Connection header received
0806 09:28:42.484196 (+ 2422us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:42.484202 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:42.484280 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:42.484437 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:28:42.486273 (+ 1836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:42.487072 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:42.489713 (+ 2641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:42.490127 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:42.490240 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:42.494581 (+ 4341us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:28:42.494586 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:28:42.494589 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:28:42.494621 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:28:42.497800 (+ 3179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:42.498479 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:42.498485 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:42.498493 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:42.498612 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:42.499094 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:42.499098 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:42.499100 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:42.499359 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:28:42.499513 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:42.500193 (+ 680us) server_negotiation.cc:299] Negotiation successful
0806 09:28:42.500455 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 09:28:42.501781 24963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:42.480660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:36461 (local address 127.20.113.2:60697)
0806 09:28:42.481600 (+ 940us) negotiation.cc:107] Waiting for socket to connect
0806 09:28:42.481650 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 09:28:42.483882 (+ 2232us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:28:42.484694 (+ 812us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:28:42.484705 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:28:42.485279 (+ 574us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:28:42.486090 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:42.486116 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:42.487253 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:42.487260 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:42.489559 (+ 2299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:42.489577 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:42.490393 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:42.490403 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:42.490729 (+ 326us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:42.491483 (+ 754us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:28:42.491524 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:28:42.494344 (+ 2820us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:28:42.497927 (+ 3583us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:28:42.497935 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:28:42.497950 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:28:42.498356 (+ 406us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:28:42.498735 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:28:42.498740 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:28:42.498744 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:28:42.498957 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:28:42.499488 (+ 531us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:28:42.499495 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:28:42.499845 (+ 350us) client_negotiation.cc:769] Sending connection context
0806 09:28:42.500106 (+ 261us) client_negotiation.cc:240] Negotiation successful
0806 09:28:42.500656 (+ 550us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":699,"thread_start_us":404,"threads_started":1}
I20260806 09:28:42.503508 24960 heartbeater.cc:347] Connected to a master server at 127.20.113.62:36461
I20260806 09:28:42.503844 24960 heartbeater.cc:464] Registering TS with master...
I20260806 09:28:42.504653 24960 heartbeater.cc:511] Master 127.20.113.62:36461 requested a full tablet report, sending...
I20260806 09:28:42.506090 24483 ts_manager.cc:194] Registered new tserver with Master: a5dd484f3d4f49f394a15251febb8a42 (127.20.113.2:36711)
I20260806 09:28:42.506846 24483 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.113.2@KRBTEST.COM'} at 127.20.113.2:60697
WARNING: no policy specified for kudu/127.20.113.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.113.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.113.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.3.keytab.
Entry for principal kudu/127.20.113.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.3.keytab.
I20260806 09:28:42.539393 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786008495763054-20932-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786008495763054-20932-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786008495763054-20932-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786008495763054-20932-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.113.3:0
--local_ip_for_outbound_sockets=127.20.113.3
--webserver_interface=127.20.113.3
--webserver_port=0
--tserver_master_addrs=127.20.113.62:36461
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.3.keytab
--principal=kudu/127.20.113.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:45167
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:28:42.874495 24967 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:28:42.874768 24967 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:28:42.874863 24967 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:28:42.882262 24967 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:28:42.882368 24967 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:28:42.882488 24967 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.113.3
I20260806 09:28:42.891927 24967 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:45167
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786008495763054-20932-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786008495763054-20932-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.3.keytab
--principal=kudu/127.20.113.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.113.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786008495763054-20932-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.113.3
--webserver_port=0
--tserver_master_addrs=127.20.113.62:36461
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.113.3
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786008495763054-20932-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:28:42.893162 24967 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:28:42.894477 24967 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:28:42.907989 24974 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:28:42.907913 24979 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:28:42.907950 24975 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:28:42.908895 24967 server_base.cc:1034] running on GCE node
I20260806 09:28:42.909499 24967 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:28:42.910374 24967 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:28:42.911602 24967 hybrid_clock.cc:648] HybridClock initialized: now 1786008522911536 us; error 53 us; skew 500 ppm
I20260806 09:28:42.911904 24967 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:28:42 dist-test-slave-45dx krb5kdc[24411](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008522, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:28:42.915777 24967 init.cc:377] Logged in from keytab as kudu/127.20.113.3@KRBTEST.COM (short username kudu)
I20260806 09:28:42.917461 24967 webserver.cc:460] Webserver started at http://127.20.113.3:36371/ using document root <none> and password file <none>
I20260806 09:28:42.918164 24967 fs_manager.cc:359] Metadata directory not provided
I20260806 09:28:42.918320 24967 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:28:42.918642 24967 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:28:42.921757 24967 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786008495763054-20932-0/minicluster-data/ts-2/data/instance:
uuid: "aab423640dc0406eb0c3e02d78dbed24"
format_stamp: "Formatted at 2026-08-06 09:28:42 on dist-test-slave-45dx"
server_key: "6fc9a2d8b779d3f4e6d24e4566d1c956"
server_key_iv: "a41c50b43a1d203e1293df86369f267b"
server_key_version: "encryptionkey@0"
I20260806 09:28:42.922585 24967 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786008495763054-20932-0/minicluster-data/ts-2/wal/instance:
uuid: "aab423640dc0406eb0c3e02d78dbed24"
format_stamp: "Formatted at 2026-08-06 09:28:42 on dist-test-slave-45dx"
server_key: "6fc9a2d8b779d3f4e6d24e4566d1c956"
server_key_iv: "a41c50b43a1d203e1293df86369f267b"
server_key_version: "encryptionkey@0"
I20260806 09:28:42.927635 24967 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260806 09:28:42.931026 24987 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:28:42.932273 24967 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 09:28:42.932518 24967 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786008495763054-20932-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786008495763054-20932-0/minicluster-data/ts-2/wal
uuid: "aab423640dc0406eb0c3e02d78dbed24"
format_stamp: "Formatted at 2026-08-06 09:28:42 on dist-test-slave-45dx"
server_key: "6fc9a2d8b779d3f4e6d24e4566d1c956"
server_key_iv: "a41c50b43a1d203e1293df86369f267b"
server_key_version: "encryptionkey@0"
I20260806 09:28:42.932724 24967 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786008495763054-20932-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786008495763054-20932-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786008495763054-20932-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:28:42.952129 24967 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:28:42.955953 24967 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:28:42.956240 24967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:28:42.957154 24967 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:28:42.958456 24967 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:28:42.958564 24967 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:28:42.958709 24967 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:28:42.958783 24967 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:28:42.993217 24967 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.3:36329
I20260806 09:28:42.993304 25167 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.3:36329 every 8 connection(s)
I20260806 09:28:42.994571 24967 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786008495763054-20932-0/minicluster-data/ts-2/data/info.pb
I20260806 09:28:42.995669 24752 heartbeater.cc:503] Master 127.20.113.62:36461 was elected leader, sending a full tablet report...
I20260806 09:28:43.000981 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 24967
I20260806 09:28:43.001191 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786008495763054-20932-0/minicluster-data/ts-2/wal/instance
I20260806 09:28:43.001655 20932 external_mini_cluster.cc:1428] Setting key 45e388f29d53f9deccf8646f4cfbe37c
Aug 06 09:28:43 dist-test-slave-45dx krb5kdc[24411](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008522, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.3@KRBTEST.COM for kudu/127.20.113.62@KRBTEST.COM
I20260806 09:28:43.014789 24754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:42.997374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.3:49013 (local address 127.20.113.62:36461)
0806 09:28:42.997598 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:42.997603 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:42.998901 (+ 1298us) server_negotiation.cc:407] Connection header received
0806 09:28:43.000959 (+ 2058us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:43.000965 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:43.001081 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:43.001186 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:28:43.003157 (+ 1971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:43.004177 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:43.005869 (+ 1692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:43.006282 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:43.006385 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:43.009766 (+ 3381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:28:43.009771 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:28:43.009774 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:28:43.009805 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:28:43.012212 (+ 2407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:43.012917 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:43.012921 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:43.012924 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:43.013059 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:43.013441 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:43.013444 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:43.013446 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:43.013693 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:28:43.013806 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:43.014402 (+ 596us) server_negotiation.cc:299] Negotiation successful
0806 09:28:43.014547 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260806 09:28:43.015890 25170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:42.997995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:36461 (local address 127.20.113.3:49013)
0806 09:28:42.998724 (+ 729us) negotiation.cc:107] Waiting for socket to connect
0806 09:28:42.998767 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 09:28:43.000615 (+ 1848us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:28:43.001436 (+ 821us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:28:43.001445 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:28:43.002082 (+ 637us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:28:43.002980 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:43.003013 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:43.004359 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:43.004366 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:43.005727 (+ 1361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:43.005740 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:43.006475 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:43.006480 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:43.006664 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:43.007280 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:28:43.007306 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:28:43.009574 (+ 2268us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:28:43.012384 (+ 2810us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:28:43.012391 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:28:43.012401 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:28:43.012810 (+ 409us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:28:43.013154 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:28:43.013157 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:28:43.013159 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:28:43.013327 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:28:43.013800 (+ 473us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:28:43.013806 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:28:43.014093 (+ 287us) client_negotiation.cc:769] Sending connection context
0806 09:28:43.014312 (+ 219us) client_negotiation.cc:240] Negotiation successful
0806 09:28:43.014672 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":293,"threads_started":1}
I20260806 09:28:43.017719 25168 heartbeater.cc:347] Connected to a master server at 127.20.113.62:36461
I20260806 09:28:43.017982 25168 heartbeater.cc:464] Registering TS with master...
I20260806 09:28:43.018646 25168 heartbeater.cc:511] Master 127.20.113.62:36461 requested a full tablet report, sending...
I20260806 09:28:43.020048 24483 ts_manager.cc:194] Registered new tserver with Master: aab423640dc0406eb0c3e02d78dbed24 (127.20.113.3:36329)
I20260806 09:28:43.020920 24483 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.113.3@KRBTEST.COM'} at 127.20.113.3:49013
I20260806 09:28:43.029482 20932 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 09:28:43.069514 24754 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 09:28:43.066972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38678 (local address 127.20.113.62:36461)
0806 09:28:43.067240 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:43.067247 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:43.067540 (+ 293us) server_negotiation.cc:407] Connection header received
0806 09:28:43.067739 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:43.067755 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:43.067829 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:43.067924 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:28:43.069253 (+ 1329us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38678: BlockingRecv error: recv got EOF from 127.0.0.1:38678 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
I20260806 09:28:43.072718 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 24577
W20260806 09:28:43.075590 20932 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 24577: Not authorized: Client connection negotiation failed: client connection to 127.20.113.1:37577: server requires authentication, but client does not have Kerberos credentials available
I20260806 09:28:43.075770 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 24577
I20260806 09:28:43.106122 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 24764
W20260806 09:28:43.108762 20932 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 24764: Not authorized: Client connection negotiation failed: client connection to 127.20.113.2:36711: server requires authentication, but client does not have Kerberos credentials available
I20260806 09:28:43.108956 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 24764
I20260806 09:28:43.131378 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 24967
W20260806 09:28:43.134558 20932 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 24967: Not authorized: Client connection negotiation failed: client connection to 127.20.113.3:36329: server requires authentication, but client does not have Kerberos credentials available
I20260806 09:28:43.134717 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 24967
I20260806 09:28:43.158677 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 24433
I20260806 09:28:43.159829 24531 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:28:43.262379 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 24433
2026-08-06T09:28:43Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (4729 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786008495763054-20932-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 09:28:43 dist-test-slave-45dx krb5kdc[25194](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 09:28:43 dist-test-slave-45dx krb5kdc[25194](info): set up 2 sockets
Aug 06 09:28:43 dist-test-slave-45dx krb5kdc[25194](info): commencing operation
krb5kdc: starting...
W20260806 09:28:44.549842 20932 mini_kdc.cc:121] Time spent starting KDC: real 1.238s user 0.000s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786008495763054-20932-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786008495763054-20932-0/krb5kdc/test-user.keytab.
Aug 06 09:28:44 dist-test-slave-45dx krb5kdc[25194](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008524, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T09:28:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T09:28:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.113.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.113.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.113.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab.
Entry for principal kudu/127.20.113.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab.
I20260806 09:28:44.783872 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786008495763054-20932-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786008495763054-20932-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786008495763054-20932-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.113.62:32979
--webserver_interface=127.20.113.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab
--principal=kudu/127.20.113.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:45115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.113.62:32979
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:28:45.116881 25215 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:28:45.117179 25215 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:28:45.117358 25215 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:28:45.124706 25215 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 09:28:45.124818 25215 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:28:45.124891 25215 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:28:45.125001 25215 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 09:28:45.125092 25215 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 09:28:45.134848 25215 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:45115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786008495763054-20932-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786008495763054-20932-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.113.62:32979
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab
--principal=kudu/127.20.113.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.113.62:32979
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.113.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786008495763054-20932-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:28:45.136198 25215 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:28:45.137519 25215 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:28:45.148962 25221 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:28:45.149145 25226 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:28:45.149170 25224 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:28:45.150341 25215 server_base.cc:1034] running on GCE node
I20260806 09:28:45.151111 25215 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:28:45.152470 25215 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:28:45.153694 25215 hybrid_clock.cc:648] HybridClock initialized: now 1786008525153664 us; error 39 us; skew 500 ppm
I20260806 09:28:45.154037 25215 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:28:45 dist-test-slave-45dx krb5kdc[25194](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008525, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:28:45.157053 25215 init.cc:377] Logged in from keytab as kudu/127.20.113.62@KRBTEST.COM (short username kudu)
I20260806 09:28:45.158440 25215 webserver.cc:460] Webserver started at http://127.20.113.62:40303/ using document root <none> and password file <none>
I20260806 09:28:45.159036 25215 fs_manager.cc:359] Metadata directory not provided
I20260806 09:28:45.159134 25215 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:28:45.159394 25215 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:28:45.162300 25215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786008495763054-20932-0/minicluster-data/master-0/data/instance:
uuid: "0b3f5b24452c47a68e388d2e1da36eb2"
format_stamp: "Formatted at 2026-08-06 09:28:45 on dist-test-slave-45dx"
server_key: "33573773abc575fc2b182aab9af01d4d"
server_key_iv: "dbe004948374b4d1152b966a1fe1ba14"
server_key_version: "encryptionkey@0"
I20260806 09:28:45.163074 25215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786008495763054-20932-0/minicluster-data/master-0/wal/instance:
uuid: "0b3f5b24452c47a68e388d2e1da36eb2"
format_stamp: "Formatted at 2026-08-06 09:28:45 on dist-test-slave-45dx"
server_key: "33573773abc575fc2b182aab9af01d4d"
server_key_iv: "dbe004948374b4d1152b966a1fe1ba14"
server_key_version: "encryptionkey@0"
I20260806 09:28:45.167980 25215 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 09:28:45.171288 25238 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:28:45.172719 25215 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 09:28:45.172855 25215 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786008495763054-20932-0/minicluster-data/master-0/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786008495763054-20932-0/minicluster-data/master-0/wal
uuid: "0b3f5b24452c47a68e388d2e1da36eb2"
format_stamp: "Formatted at 2026-08-06 09:28:45 on dist-test-slave-45dx"
server_key: "33573773abc575fc2b182aab9af01d4d"
server_key_iv: "dbe004948374b4d1152b966a1fe1ba14"
server_key_version: "encryptionkey@0"
I20260806 09:28:45.173025 25215 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786008495763054-20932-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786008495763054-20932-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786008495763054-20932-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:28:45.200042 25215 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:28:45.204075 25215 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:28:45.204336 25215 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:28:45.222960 25215 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.62:32979
I20260806 09:28:45.222990 25313 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.62:32979 every 8 connection(s)
I20260806 09:28:45.224400 25215 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
I20260806 09:28:45.228164 25316 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:28:45.234810 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 25215
I20260806 09:28:45.235086 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786008495763054-20932-0/minicluster-data/master-0/wal/instance
I20260806 09:28:45.235978 20932 external_mini_cluster.cc:1428] Setting key 197d1d5981ef5fd601320081b0da3767
I20260806 09:28:45.239589 25316 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b3f5b24452c47a68e388d2e1da36eb2: Bootstrap starting.
I20260806 09:28:45.243360 25316 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0b3f5b24452c47a68e388d2e1da36eb2: Neither blocks nor log segments found. Creating new log.
I20260806 09:28:45.244560 25316 log.cc:824] T 00000000000000000000000000000000 P 0b3f5b24452c47a68e388d2e1da36eb2: Log is configured to *not* fsync() on all Append() calls
I20260806 09:28:45.247982 25316 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b3f5b24452c47a68e388d2e1da36eb2: No bootstrap required, opened a new log
Aug 06 09:28:45 dist-test-slave-45dx krb5kdc[25194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008524, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.113.62@KRBTEST.COM
I20260806 09:28:45.257179 25316 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0b3f5b24452c47a68e388d2e1da36eb2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b3f5b24452c47a68e388d2e1da36eb2" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 32979 } }
I20260806 09:28:45.257587 25316 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0b3f5b24452c47a68e388d2e1da36eb2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:28:45.257699 25316 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0b3f5b24452c47a68e388d2e1da36eb2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b3f5b24452c47a68e388d2e1da36eb2, State: Initialized, Role: FOLLOWER
I20260806 09:28:45.258440 25316 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0b3f5b24452c47a68e388d2e1da36eb2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b3f5b24452c47a68e388d2e1da36eb2" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 32979 } }
I20260806 09:28:45.258744 25316 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0b3f5b24452c47a68e388d2e1da36eb2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 09:28:45.258898 25316 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0b3f5b24452c47a68e388d2e1da36eb2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 09:28:45.259095 25316 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0b3f5b24452c47a68e388d2e1da36eb2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:28:45.260018 25321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:45.237970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54648 (local address 127.20.113.62:32979)
0806 09:28:45.238713 (+ 743us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:45.238725 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:45.238765 (+ 40us) server_negotiation.cc:407] Connection header received
0806 09:28:45.240602 (+ 1837us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:45.240623 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:45.240994 (+ 371us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:45.241320 (+ 326us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:28:45.243386 (+ 2066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:45.244930 (+ 1544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:45.246808 (+ 1878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:45.247835 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:45.248045 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:45.252558 (+ 4513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:28:45.252566 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:28:45.252581 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:28:45.252624 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:28:45.255728 (+ 3104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:45.256563 (+ 835us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:45.256569 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:45.256575 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:45.256734 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:45.257140 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:45.257143 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:45.257145 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:45.257827 (+ 682us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:28:45.258137 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:45.258506 (+ 369us) server_negotiation.cc:299] Negotiation successful
0806 09:28:45.258933 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":529,"thread_start_us":332,"threads_started":1}
I20260806 09:28:45.261991 25316 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0b3f5b24452c47a68e388d2e1da36eb2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b3f5b24452c47a68e388d2e1da36eb2" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 32979 } }
I20260806 09:28:45.262585 25316 leader_election.cc:302] T 00000000000000000000000000000000 P 0b3f5b24452c47a68e388d2e1da36eb2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0b3f5b24452c47a68e388d2e1da36eb2; no voters:
I20260806 09:28:45.263249 25316 leader_election.cc:288] T 00000000000000000000000000000000 P 0b3f5b24452c47a68e388d2e1da36eb2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 09:28:45.263383 25324 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0b3f5b24452c47a68e388d2e1da36eb2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:28:45.264390 20932 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 09:28:45.264714 25324 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0b3f5b24452c47a68e388d2e1da36eb2 [term 1 LEADER]: Becoming Leader. State: Replica: 0b3f5b24452c47a68e388d2e1da36eb2, State: Running, Role: LEADER
I20260806 09:28:45.265203 25324 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0b3f5b24452c47a68e388d2e1da36eb2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b3f5b24452c47a68e388d2e1da36eb2" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 32979 } }
I20260806 09:28:45.265511 25316 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0b3f5b24452c47a68e388d2e1da36eb2 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 09:28:45.267823 25326 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b3f5b24452c47a68e388d2e1da36eb2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0b3f5b24452c47a68e388d2e1da36eb2. Latest consensus state: current_term: 1 leader_uuid: "0b3f5b24452c47a68e388d2e1da36eb2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b3f5b24452c47a68e388d2e1da36eb2" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 32979 } } }
I20260806 09:28:45.268190 25326 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b3f5b24452c47a68e388d2e1da36eb2 [sys.catalog]: This master's current role is: LEADER
I20260806 09:28:45.268646 25325 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b3f5b24452c47a68e388d2e1da36eb2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0b3f5b24452c47a68e388d2e1da36eb2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b3f5b24452c47a68e388d2e1da36eb2" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 32979 } } }
I20260806 09:28:45.268929 25325 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b3f5b24452c47a68e388d2e1da36eb2 [sys.catalog]: This master's current role is: LEADER
I20260806 09:28:45.272094 25331 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 09:28:45.277117 25331 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 09:28:45.287412 25331 catalog_manager.cc:1347] Generated new cluster ID: f7eacdb631f44b709429433f6ca4a6b1
I20260806 09:28:45.287556 25331 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 09:28:45.297705 25331 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 09:28:45.299448 25331 catalog_manager.cc:1504] Loading token signing keys...
I20260806 09:28:45.311255 25331 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0b3f5b24452c47a68e388d2e1da36eb2: Generated new TSK 0
I20260806 09:28:45.312479 25331 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 09:28:45.324209 25321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:45.315963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54650 (local address 127.20.113.62:32979)
0806 09:28:45.316174 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:45.316180 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:45.316634 (+ 454us) server_negotiation.cc:407] Connection header received
0806 09:28:45.316780 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:45.316786 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:45.316894 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:45.317017 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:28:45.317969 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:45.319122 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:45.320520 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:45.321292 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:45.321450 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:45.322522 (+ 1072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:28:45.322529 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:28:45.322534 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:28:45.323415 (+ 881us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:28:45.323489 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:28:45.323523 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:28:45.323646 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:45.323729 (+ 83us) server_negotiation.cc:299] Negotiation successful
0806 09:28:45.323827 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 09:28:45.331707 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 25215
I20260806 09:28:45.333043 25307 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:28:45.422892 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 25215
2026-08-06T09:28:45Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (2154 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786008495763054-20932-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 09:28:45 dist-test-slave-45dx krb5kdc[25360](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 09:28:45 dist-test-slave-45dx krb5kdc[25360](info): set up 2 sockets
Aug 06 09:28:45 dist-test-slave-45dx krb5kdc[25360](info): commencing operation
krb5kdc: starting...
W20260806 09:28:46.685186 20932 mini_kdc.cc:121] Time spent starting KDC: real 1.218s user 0.000s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786008495763054-20932-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786008495763054-20932-0/krb5kdc/test-user.keytab.
Aug 06 09:28:46 dist-test-slave-45dx krb5kdc[25360](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008526, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T09:28:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T09:28:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.113.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.113.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.113.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab.
Entry for principal kudu/127.20.113.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab.
I20260806 09:28:46.909991 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786008495763054-20932-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786008495763054-20932-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786008495763054-20932-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.113.62:37765
--webserver_interface=127.20.113.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab
--principal=kudu/127.20.113.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:32911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.113.62:37765
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:28:47.237309 25377 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:28:47.237578 25377 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:28:47.237727 25377 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:28:47.244686 25377 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 09:28:47.244786 25377 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:28:47.244908 25377 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:28:47.245014 25377 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 09:28:47.245095 25377 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 09:28:47.254488 25377 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:32911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786008495763054-20932-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786008495763054-20932-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.113.62:37765
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab
--principal=kudu/127.20.113.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.113.62:37765
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.113.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786008495763054-20932-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:28:47.255795 25377 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:28:47.257120 25377 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:28:47.269356 25385 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:28:47.269361 25386 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:28:47.270011 25377 server_base.cc:1034] running on GCE node
W20260806 09:28:47.270089 25388 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:28:47.270910 25377 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:28:47.272377 25377 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:28:47.273627 25377 hybrid_clock.cc:648] HybridClock initialized: now 1786008527273584 us; error 55 us; skew 500 ppm
I20260806 09:28:47.273972 25377 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:28:47 dist-test-slave-45dx krb5kdc[25360](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008527, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:28:47.277254 25377 init.cc:377] Logged in from keytab as kudu/127.20.113.62@KRBTEST.COM (short username kudu)
I20260806 09:28:47.278749 25377 webserver.cc:460] Webserver started at http://127.20.113.62:46167/ using document root <none> and password file <none>
I20260806 09:28:47.279503 25377 fs_manager.cc:359] Metadata directory not provided
I20260806 09:28:47.279649 25377 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:28:47.279970 25377 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:28:47.283034 25377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786008495763054-20932-0/minicluster-data/master-0/data/instance:
uuid: "4c7d8c8b689a48b9a2815fad4595cdf0"
format_stamp: "Formatted at 2026-08-06 09:28:47 on dist-test-slave-45dx"
server_key: "81a294ac6dfa656a536f185835b78d16"
server_key_iv: "b4764d17a4cdb23f07f5fcd20f24ea82"
server_key_version: "encryptionkey@0"
I20260806 09:28:47.283814 25377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786008495763054-20932-0/minicluster-data/master-0/wal/instance:
uuid: "4c7d8c8b689a48b9a2815fad4595cdf0"
format_stamp: "Formatted at 2026-08-06 09:28:47 on dist-test-slave-45dx"
server_key: "81a294ac6dfa656a536f185835b78d16"
server_key_iv: "b4764d17a4cdb23f07f5fcd20f24ea82"
server_key_version: "encryptionkey@0"
I20260806 09:28:47.289041 25377 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260806 09:28:47.292611 25398 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:28:47.293803 25377 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 09:28:47.293979 25377 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786008495763054-20932-0/minicluster-data/master-0/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786008495763054-20932-0/minicluster-data/master-0/wal
uuid: "4c7d8c8b689a48b9a2815fad4595cdf0"
format_stamp: "Formatted at 2026-08-06 09:28:47 on dist-test-slave-45dx"
server_key: "81a294ac6dfa656a536f185835b78d16"
server_key_iv: "b4764d17a4cdb23f07f5fcd20f24ea82"
server_key_version: "encryptionkey@0"
I20260806 09:28:47.294173 25377 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786008495763054-20932-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786008495763054-20932-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786008495763054-20932-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:28:47.322230 25377 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:28:47.326601 25377 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:28:47.326901 25377 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:28:47.346138 25377 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.62:37765
I20260806 09:28:47.346166 25472 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.62:37765 every 8 connection(s)
I20260806 09:28:47.347560 25377 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
I20260806 09:28:47.350960 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 25377
I20260806 09:28:47.351187 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786008495763054-20932-0/minicluster-data/master-0/wal/instance
I20260806 09:28:47.351347 25474 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:28:47.351953 20932 external_mini_cluster.cc:1428] Setting key ab88be8647d04f40794532721f9da73c
I20260806 09:28:47.365271 25474 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c7d8c8b689a48b9a2815fad4595cdf0: Bootstrap starting.
Aug 06 09:28:47 dist-test-slave-45dx krb5kdc[25360](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008526, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.113.62@KRBTEST.COM
I20260806 09:28:47.368903 25474 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4c7d8c8b689a48b9a2815fad4595cdf0: Neither blocks nor log segments found. Creating new log.
I20260806 09:28:47.370136 25474 log.cc:824] T 00000000000000000000000000000000 P 4c7d8c8b689a48b9a2815fad4595cdf0: Log is configured to *not* fsync() on all Append() calls
I20260806 09:28:47.373162 25474 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c7d8c8b689a48b9a2815fad4595cdf0: No bootstrap required, opened a new log
I20260806 09:28:47.375653 25477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:47.354226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57170 (local address 127.20.113.62:37765)
0806 09:28:47.355075 (+ 849us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:47.355086 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:47.355129 (+ 43us) server_negotiation.cc:407] Connection header received
0806 09:28:47.357335 (+ 2206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:47.357367 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:47.357750 (+ 383us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:47.358070 (+ 320us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:28:47.359786 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:47.361158 (+ 1372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:47.362955 (+ 1797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:47.363585 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:47.363723 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:47.368498 (+ 4775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:28:47.368505 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:28:47.368519 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:28:47.368554 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:28:47.371677 (+ 3123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:47.372387 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:47.372402 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:47.372408 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:47.372569 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:47.372950 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:47.372955 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:47.372960 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:47.373494 (+ 534us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:28:47.373830 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:47.374246 (+ 416us) server_negotiation.cc:299] Negotiation successful
0806 09:28:47.374672 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":21,"server-negotiator.queue_time_us":639,"thread_start_us":403,"threads_started":1}
I20260806 09:28:47.380921 25474 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4c7d8c8b689a48b9a2815fad4595cdf0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c7d8c8b689a48b9a2815fad4595cdf0" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 37765 } }
I20260806 09:28:47.381283 25474 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4c7d8c8b689a48b9a2815fad4595cdf0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:28:47.381397 25474 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4c7d8c8b689a48b9a2815fad4595cdf0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c7d8c8b689a48b9a2815fad4595cdf0, State: Initialized, Role: FOLLOWER
I20260806 09:28:47.381937 25474 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4c7d8c8b689a48b9a2815fad4595cdf0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c7d8c8b689a48b9a2815fad4595cdf0" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 37765 } }
I20260806 09:28:47.382220 25474 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4c7d8c8b689a48b9a2815fad4595cdf0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 09:28:47.382373 25474 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4c7d8c8b689a48b9a2815fad4595cdf0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 09:28:47.382544 25474 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4c7d8c8b689a48b9a2815fad4595cdf0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:28:47.384632 25474 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4c7d8c8b689a48b9a2815fad4595cdf0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c7d8c8b689a48b9a2815fad4595cdf0" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 37765 } }
I20260806 09:28:47.385071 25474 leader_election.cc:302] T 00000000000000000000000000000000 P 4c7d8c8b689a48b9a2815fad4595cdf0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4c7d8c8b689a48b9a2815fad4595cdf0; no voters:
I20260806 09:28:47.385674 25474 leader_election.cc:288] T 00000000000000000000000000000000 P 4c7d8c8b689a48b9a2815fad4595cdf0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 09:28:47.385809 25479 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4c7d8c8b689a48b9a2815fad4595cdf0 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:28:47.386562 25479 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4c7d8c8b689a48b9a2815fad4595cdf0 [term 1 LEADER]: Becoming Leader. State: Replica: 4c7d8c8b689a48b9a2815fad4595cdf0, State: Running, Role: LEADER
I20260806 09:28:47.387167 25479 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4c7d8c8b689a48b9a2815fad4595cdf0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c7d8c8b689a48b9a2815fad4595cdf0" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 37765 } }
I20260806 09:28:47.387872 25474 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4c7d8c8b689a48b9a2815fad4595cdf0 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 09:28:47.390506 25483 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c7d8c8b689a48b9a2815fad4595cdf0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4c7d8c8b689a48b9a2815fad4595cdf0. Latest consensus state: current_term: 1 leader_uuid: "4c7d8c8b689a48b9a2815fad4595cdf0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c7d8c8b689a48b9a2815fad4595cdf0" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 37765 } } }
I20260806 09:28:47.391000 25483 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c7d8c8b689a48b9a2815fad4595cdf0 [sys.catalog]: This master's current role is: LEADER
I20260806 09:28:47.391476 25482 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c7d8c8b689a48b9a2815fad4595cdf0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4c7d8c8b689a48b9a2815fad4595cdf0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c7d8c8b689a48b9a2815fad4595cdf0" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 37765 } } }
I20260806 09:28:47.391813 25482 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c7d8c8b689a48b9a2815fad4595cdf0 [sys.catalog]: This master's current role is: LEADER
I20260806 09:28:47.397732 25492 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 09:28:47.407539 25492 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 09:28:47.415719 25492 catalog_manager.cc:1347] Generated new cluster ID: 31b7a7e652da42308782c7c126218ff4
I20260806 09:28:47.415884 25492 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 09:28:47.430593 25492 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 09:28:47.431861 25492 catalog_manager.cc:1504] Loading token signing keys...
I20260806 09:28:47.440255 25492 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4c7d8c8b689a48b9a2815fad4595cdf0: Generated new TSK 0
I20260806 09:28:47.441084 25492 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.113.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.113.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.113.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.1.keytab.
Entry for principal kudu/127.20.113.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.1.keytab.
I20260806 09:28:47.504123 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786008495763054-20932-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786008495763054-20932-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786008495763054-20932-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786008495763054-20932-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.113.1:0
--local_ip_for_outbound_sockets=127.20.113.1
--webserver_interface=127.20.113.1
--webserver_port=0
--tserver_master_addrs=127.20.113.62:37765
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.1.keytab
--principal=kudu/127.20.113.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:32911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:28:47.842928 25510 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:28:47.843195 25510 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:28:47.843288 25510 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:28:47.850379 25510 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:28:47.850484 25510 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:28:47.850593 25510 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.113.1
I20260806 09:28:47.860140 25510 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:32911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786008495763054-20932-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786008495763054-20932-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.1.keytab
--principal=kudu/127.20.113.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.113.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786008495763054-20932-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.113.1
--webserver_port=0
--tserver_master_addrs=127.20.113.62:37765
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.113.1
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786008495763054-20932-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:28:47.861398 25510 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:28:47.862680 25510 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:28:47.876376 25517 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:28:47.876466 25516 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:28:47.876492 25520 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:28:47.877539 25510 server_base.cc:1034] running on GCE node
I20260806 09:28:47.878124 25510 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:28:47.879026 25510 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:28:47.880251 25510 hybrid_clock.cc:648] HybridClock initialized: now 1786008527880211 us; error 52 us; skew 500 ppm
I20260806 09:28:47.880630 25510 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:28:47 dist-test-slave-45dx krb5kdc[25360](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008527, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:28:47.883723 25510 init.cc:377] Logged in from keytab as kudu/127.20.113.1@KRBTEST.COM (short username kudu)
I20260806 09:28:47.885174 25510 webserver.cc:460] Webserver started at http://127.20.113.1:41757/ using document root <none> and password file <none>
I20260806 09:28:47.885874 25510 fs_manager.cc:359] Metadata directory not provided
I20260806 09:28:47.886026 25510 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:28:47.886340 25510 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:28:47.889390 25510 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786008495763054-20932-0/minicluster-data/ts-0/data/instance:
uuid: "355a00de68fb4b03b37554f20a8a4942"
format_stamp: "Formatted at 2026-08-06 09:28:47 on dist-test-slave-45dx"
server_key: "9c0dcdcb583be796c2713278bcfdd344"
server_key_iv: "f50a08b26949ae4cf041c38b00d6d8ab"
server_key_version: "encryptionkey@0"
I20260806 09:28:47.890180 25510 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786008495763054-20932-0/minicluster-data/ts-0/wal/instance:
uuid: "355a00de68fb4b03b37554f20a8a4942"
format_stamp: "Formatted at 2026-08-06 09:28:47 on dist-test-slave-45dx"
server_key: "9c0dcdcb583be796c2713278bcfdd344"
server_key_iv: "f50a08b26949ae4cf041c38b00d6d8ab"
server_key_version: "encryptionkey@0"
I20260806 09:28:47.895228 25510 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260806 09:28:47.899013 25530 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:28:47.900245 25510 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260806 09:28:47.900468 25510 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786008495763054-20932-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786008495763054-20932-0/minicluster-data/ts-0/wal
uuid: "355a00de68fb4b03b37554f20a8a4942"
format_stamp: "Formatted at 2026-08-06 09:28:47 on dist-test-slave-45dx"
server_key: "9c0dcdcb583be796c2713278bcfdd344"
server_key_iv: "f50a08b26949ae4cf041c38b00d6d8ab"
server_key_version: "encryptionkey@0"
I20260806 09:28:47.900664 25510 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786008495763054-20932-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786008495763054-20932-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786008495763054-20932-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:28:47.914994 25510 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:28:47.919080 25510 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:28:47.919421 25510 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:28:47.920490 25510 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:28:47.921926 25510 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:28:47.922044 25510 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:28:47.922199 25510 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:28:47.922281 25510 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:28:47.958959 25510 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.1:40341
I20260806 09:28:47.959002 25684 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.1:40341 every 8 connection(s)
I20260806 09:28:47.960297 25510 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786008495763054-20932-0/minicluster-data/ts-0/data/info.pb
I20260806 09:28:47.969092 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 25510
I20260806 09:28:47.969324 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786008495763054-20932-0/minicluster-data/ts-0/wal/instance
I20260806 09:28:47.970012 20932 external_mini_cluster.cc:1428] Setting key b627e7e17211cdbce85b185296d7f96e
Aug 06 09:28:47 dist-test-slave-45dx krb5kdc[25360](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008527, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.1@KRBTEST.COM for kudu/127.20.113.62@KRBTEST.COM
I20260806 09:28:47.981673 25690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:47.962893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.1:41123 (local address 127.20.113.62:37765)
0806 09:28:47.963368 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:47.963372 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:47.964234 (+ 862us) server_negotiation.cc:407] Connection header received
0806 09:28:47.966156 (+ 1922us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:47.966162 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:47.966258 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:47.966385 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:28:47.967760 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:47.968483 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:47.970137 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:47.970523 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:47.970663 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:47.974250 (+ 3587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:28:47.974258 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:28:47.974265 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:28:47.974310 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:28:47.978020 (+ 3710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:47.978922 (+ 902us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:47.978930 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:47.978936 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:47.979094 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:47.979560 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:47.979567 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:47.979573 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:47.979941 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:28:47.980094 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:47.980928 (+ 834us) server_negotiation.cc:299] Negotiation successful
0806 09:28:47.981180 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":361,"thread_start_us":259,"threads_started":1}
I20260806 09:28:47.982708 25691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:47.963349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:37765 (local address 127.20.113.1:41123)
0806 09:28:47.964083 (+ 734us) negotiation.cc:107] Waiting for socket to connect
0806 09:28:47.964120 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 09:28:47.965900 (+ 1780us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:28:47.966609 (+ 709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:28:47.966616 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:28:47.967100 (+ 484us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:28:47.967602 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:47.967616 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:47.968635 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:47.968639 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:47.969975 (+ 1336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:47.969988 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:47.970753 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:47.970763 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:47.971008 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:47.971696 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:28:47.971718 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:28:47.974029 (+ 2311us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:28:47.978185 (+ 4156us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:28:47.978195 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:28:47.978213 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:28:47.978757 (+ 544us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:28:47.979209 (+ 452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:28:47.979216 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:28:47.979222 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:28:47.979447 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:28:47.980070 (+ 623us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:28:47.980080 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:28:47.980505 (+ 425us) client_negotiation.cc:769] Sending connection context
0806 09:28:47.980792 (+ 287us) client_negotiation.cc:240] Negotiation successful
0806 09:28:47.981315 (+ 523us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":518,"thread_start_us":302,"threads_started":1}
I20260806 09:28:47.984401 25686 heartbeater.cc:347] Connected to a master server at 127.20.113.62:37765
I20260806 09:28:47.984732 25686 heartbeater.cc:464] Registering TS with master...
I20260806 09:28:47.985471 25686 heartbeater.cc:511] Master 127.20.113.62:37765 requested a full tablet report, sending...
I20260806 09:28:47.987543 25423 ts_manager.cc:194] Registered new tserver with Master: 355a00de68fb4b03b37554f20a8a4942 (127.20.113.1:40341)
I20260806 09:28:47.989073 25423 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.113.1@KRBTEST.COM'} at 127.20.113.1:41123
WARNING: no policy specified for kudu/127.20.113.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.113.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.113.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.2.keytab.
Entry for principal kudu/127.20.113.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.2.keytab.
I20260806 09:28:48.026940 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786008495763054-20932-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786008495763054-20932-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786008495763054-20932-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786008495763054-20932-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.113.2:0
--local_ip_for_outbound_sockets=127.20.113.2
--webserver_interface=127.20.113.2
--webserver_port=0
--tserver_master_addrs=127.20.113.62:37765
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.2.keytab
--principal=kudu/127.20.113.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:32911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:28:48.368579 25698 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:28:48.368988 25698 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:28:48.369176 25698 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:28:48.378646 25698 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:28:48.378778 25698 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:28:48.378913 25698 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.113.2
I20260806 09:28:48.390267 25698 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:32911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786008495763054-20932-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786008495763054-20932-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.2.keytab
--principal=kudu/127.20.113.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.113.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786008495763054-20932-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.113.2
--webserver_port=0
--tserver_master_addrs=127.20.113.62:37765
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.113.2
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786008495763054-20932-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:28:48.391532 25698 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:28:48.393047 25698 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:28:48.407195 25706 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:28:48.407140 25704 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:28:48.407668 25703 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:28:48.408473 25698 server_base.cc:1034] running on GCE node
I20260806 09:28:48.409158 25698 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:28:48.410133 25698 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:28:48.411403 25698 hybrid_clock.cc:648] HybridClock initialized: now 1786008528411307 us; error 80 us; skew 500 ppm
I20260806 09:28:48.411749 25698 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:28:48 dist-test-slave-45dx krb5kdc[25360](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008528, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:28:48.419291 25698 init.cc:377] Logged in from keytab as kudu/127.20.113.2@KRBTEST.COM (short username kudu)
I20260806 09:28:48.421017 25698 webserver.cc:460] Webserver started at http://127.20.113.2:44679/ using document root <none> and password file <none>
I20260806 09:28:48.421798 25698 fs_manager.cc:359] Metadata directory not provided
I20260806 09:28:48.421965 25698 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:28:48.422314 25698 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:28:48.425551 25698 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786008495763054-20932-0/minicluster-data/ts-1/data/instance:
uuid: "c96f19c9865943f9b0eb0406d816e755"
format_stamp: "Formatted at 2026-08-06 09:28:48 on dist-test-slave-45dx"
server_key: "0486b8d8e2cb3b0adc6fd46de171a957"
server_key_iv: "4d2beb1108b715f2349d96a9e894c30d"
server_key_version: "encryptionkey@0"
I20260806 09:28:48.426420 25698 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786008495763054-20932-0/minicluster-data/ts-1/wal/instance:
uuid: "c96f19c9865943f9b0eb0406d816e755"
format_stamp: "Formatted at 2026-08-06 09:28:48 on dist-test-slave-45dx"
server_key: "0486b8d8e2cb3b0adc6fd46de171a957"
server_key_iv: "4d2beb1108b715f2349d96a9e894c30d"
server_key_version: "encryptionkey@0"
I20260806 09:28:48.431727 25698 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260806 09:28:48.435475 25715 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:28:48.436889 25698 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 09:28:48.437072 25698 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786008495763054-20932-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786008495763054-20932-0/minicluster-data/ts-1/wal
uuid: "c96f19c9865943f9b0eb0406d816e755"
format_stamp: "Formatted at 2026-08-06 09:28:48 on dist-test-slave-45dx"
server_key: "0486b8d8e2cb3b0adc6fd46de171a957"
server_key_iv: "4d2beb1108b715f2349d96a9e894c30d"
server_key_version: "encryptionkey@0"
I20260806 09:28:48.437280 25698 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786008495763054-20932-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786008495763054-20932-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786008495763054-20932-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:28:48.462026 25698 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:28:48.466697 25698 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:28:48.467021 25698 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:28:48.468103 25698 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:28:48.469584 25698 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:28:48.469698 25698 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:28:48.469863 25698 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:28:48.469949 25698 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:28:48.505519 25698 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.2:41775
I20260806 09:28:48.505580 25866 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.2:41775 every 8 connection(s)
I20260806 09:28:48.506882 25698 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786008495763054-20932-0/minicluster-data/ts-1/data/info.pb
I20260806 09:28:48.508527 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 25698
I20260806 09:28:48.508733 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786008495763054-20932-0/minicluster-data/ts-1/wal/instance
I20260806 09:28:48.509336 20932 external_mini_cluster.cc:1428] Setting key 2eac92f2c8e11120f645fe47cb5b837d
Aug 06 09:28:48 dist-test-slave-45dx krb5kdc[25360](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008528, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.2@KRBTEST.COM for kudu/127.20.113.62@KRBTEST.COM
I20260806 09:28:48.528055 25874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:48.509910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.2:33393 (local address 127.20.113.62:37765)
0806 09:28:48.510420 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:48.510424 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:48.511663 (+ 1239us) server_negotiation.cc:407] Connection header received
0806 09:28:48.513565 (+ 1902us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:48.513570 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:48.513650 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:48.513770 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:28:48.515199 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:48.515908 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:48.517548 (+ 1640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:48.517980 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:48.518123 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:48.521937 (+ 3814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:28:48.521942 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:28:48.521945 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:28:48.521984 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:28:48.524816 (+ 2832us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:48.525743 (+ 927us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:48.525747 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:48.525749 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:48.525902 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:48.526383 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:48.526386 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:48.526387 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:48.526628 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:28:48.526733 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:48.527615 (+ 882us) server_negotiation.cc:299] Negotiation successful
0806 09:28:48.527771 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":406,"thread_start_us":280,"threads_started":1}
I20260806 09:28:48.529577 25877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:48.510584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:37765 (local address 127.20.113.2:33393)
0806 09:28:48.511491 (+ 907us) negotiation.cc:107] Waiting for socket to connect
0806 09:28:48.511544 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 09:28:48.513314 (+ 1770us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:28:48.514007 (+ 693us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:28:48.514015 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:28:48.514500 (+ 485us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:28:48.515035 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:48.515051 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:48.516061 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:48.516065 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:48.517395 (+ 1330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:48.517407 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:48.518218 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:48.518225 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:48.518461 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:48.519151 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:28:48.519172 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:28:48.521643 (+ 2471us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:28:48.524961 (+ 3318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:28:48.524973 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:28:48.524990 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:28:48.525616 (+ 626us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:28:48.526010 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:28:48.526017 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:28:48.526023 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:28:48.526265 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:28:48.526765 (+ 500us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:28:48.526776 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:28:48.527229 (+ 453us) client_negotiation.cc:769] Sending connection context
0806 09:28:48.527534 (+ 305us) client_negotiation.cc:240] Negotiation successful
0806 09:28:48.528077 (+ 543us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":606,"thread_start_us":317,"threads_started":1}
I20260806 09:28:48.531129 25868 heartbeater.cc:347] Connected to a master server at 127.20.113.62:37765
I20260806 09:28:48.531479 25868 heartbeater.cc:464] Registering TS with master...
I20260806 09:28:48.532428 25868 heartbeater.cc:511] Master 127.20.113.62:37765 requested a full tablet report, sending...
I20260806 09:28:48.534055 25423 ts_manager.cc:194] Registered new tserver with Master: c96f19c9865943f9b0eb0406d816e755 (127.20.113.2:41775)
I20260806 09:28:48.534857 25423 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.113.2@KRBTEST.COM'} at 127.20.113.2:33393
WARNING: no policy specified for kudu/127.20.113.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.113.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.113.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.3.keytab.
Entry for principal kudu/127.20.113.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.3.keytab.
I20260806 09:28:48.567596 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786008495763054-20932-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786008495763054-20932-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786008495763054-20932-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786008495763054-20932-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.113.3:0
--local_ip_for_outbound_sockets=127.20.113.3
--webserver_interface=127.20.113.3
--webserver_port=0
--tserver_master_addrs=127.20.113.62:37765
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.3.keytab
--principal=kudu/127.20.113.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:32911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:28:48.905865 25880 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:28:48.906205 25880 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:28:48.906388 25880 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:28:48.913897 25880 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:28:48.914060 25880 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:28:48.914242 25880 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.113.3
I20260806 09:28:48.924081 25880 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:32911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786008495763054-20932-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786008495763054-20932-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.3.keytab
--principal=kudu/127.20.113.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.113.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786008495763054-20932-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.113.3
--webserver_port=0
--tserver_master_addrs=127.20.113.62:37765
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.113.3
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786008495763054-20932-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:28:48.925547 25880 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:28:48.926914 25880 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:28:48.940734 25890 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:28:48.940888 25888 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:28:48.941815 25893 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:28:48.942529 25880 server_base.cc:1034] running on GCE node
I20260806 09:28:48.943131 25880 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:28:48.944073 25880 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:28:48.945304 25880 hybrid_clock.cc:648] HybridClock initialized: now 1786008528945240 us; error 74 us; skew 500 ppm
I20260806 09:28:48.945634 25880 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:28:48 dist-test-slave-45dx krb5kdc[25360](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008528, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:28:48.948755 25880 init.cc:377] Logged in from keytab as kudu/127.20.113.3@KRBTEST.COM (short username kudu)
I20260806 09:28:48.950232 25880 webserver.cc:460] Webserver started at http://127.20.113.3:33683/ using document root <none> and password file <none>
I20260806 09:28:48.950927 25880 fs_manager.cc:359] Metadata directory not provided
I20260806 09:28:48.951074 25880 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:28:48.951400 25880 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:28:48.954397 25880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786008495763054-20932-0/minicluster-data/ts-2/data/instance:
uuid: "afeb1420748548cb8c1ee20198d5542d"
format_stamp: "Formatted at 2026-08-06 09:28:48 on dist-test-slave-45dx"
server_key: "db146299faaa6a3eba43eb483121e50f"
server_key_iv: "cf4c3776f8448cdf858ccb74408a549f"
server_key_version: "encryptionkey@0"
I20260806 09:28:48.955202 25880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786008495763054-20932-0/minicluster-data/ts-2/wal/instance:
uuid: "afeb1420748548cb8c1ee20198d5542d"
format_stamp: "Formatted at 2026-08-06 09:28:48 on dist-test-slave-45dx"
server_key: "db146299faaa6a3eba43eb483121e50f"
server_key_iv: "cf4c3776f8448cdf858ccb74408a549f"
server_key_version: "encryptionkey@0"
I20260806 09:28:48.960747 25880 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.003s
I20260806 09:28:48.964414 25905 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:28:48.965703 25880 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 09:28:48.965901 25880 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786008495763054-20932-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786008495763054-20932-0/minicluster-data/ts-2/wal
uuid: "afeb1420748548cb8c1ee20198d5542d"
format_stamp: "Formatted at 2026-08-06 09:28:48 on dist-test-slave-45dx"
server_key: "db146299faaa6a3eba43eb483121e50f"
server_key_iv: "cf4c3776f8448cdf858ccb74408a549f"
server_key_version: "encryptionkey@0"
I20260806 09:28:48.966125 25880 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786008495763054-20932-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786008495763054-20932-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786008495763054-20932-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:28:48.979435 25880 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:28:48.983283 25880 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:28:48.983597 25880 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:28:48.984576 25880 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:28:48.985906 25880 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:28:48.986012 25880 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:28:48.986164 25880 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:28:48.986244 25880 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:28:48.991868 25686 heartbeater.cc:503] Master 127.20.113.62:37765 was elected leader, sending a full tablet report...
I20260806 09:28:49.021848 25880 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.3:42463
I20260806 09:28:49.021870 26060 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.3:42463 every 8 connection(s)
I20260806 09:28:49.023172 25880 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786008495763054-20932-0/minicluster-data/ts-2/data/info.pb
I20260806 09:28:49.031032 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 25880
I20260806 09:28:49.031256 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786008495763054-20932-0/minicluster-data/ts-2/wal/instance
I20260806 09:28:49.031733 20932 external_mini_cluster.cc:1428] Setting key f13e48b3d08040149069c1621b0bcf25
Aug 06 09:28:49 dist-test-slave-45dx krb5kdc[25360](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008528, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.3@KRBTEST.COM for kudu/127.20.113.62@KRBTEST.COM
I20260806 09:28:49.044661 25874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:49.025838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.3:41347 (local address 127.20.113.62:37765)
0806 09:28:49.026030 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:49.026041 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:49.027188 (+ 1147us) server_negotiation.cc:407] Connection header received
0806 09:28:49.029168 (+ 1980us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:49.029174 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:49.029285 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:49.029404 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:28:49.031111 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:49.032001 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:49.034233 (+ 2232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:49.034664 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:49.034800 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:49.038475 (+ 3675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:28:49.038480 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:28:49.038483 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:28:49.038517 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:28:49.041448 (+ 2931us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:49.042235 (+ 787us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:49.042240 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:49.042242 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:49.042375 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:49.042846 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:49.042850 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:49.042854 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:49.043137 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:28:49.043306 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:49.044007 (+ 701us) server_negotiation.cc:299] Negotiation successful
0806 09:28:49.044246 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260806 09:28:49.045879 26064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:49.026293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:37765 (local address 127.20.113.3:41347)
0806 09:28:49.027021 (+ 728us) negotiation.cc:107] Waiting for socket to connect
0806 09:28:49.027067 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 09:28:49.028887 (+ 1820us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:28:49.029651 (+ 764us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:28:49.029660 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:28:49.030175 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:28:49.030896 (+ 721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:49.030923 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:49.032158 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:49.032164 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:49.034090 (+ 1926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:49.034103 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:49.034893 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:49.034901 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:49.035144 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:49.035819 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:28:49.035843 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:28:49.038246 (+ 2403us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:28:49.041602 (+ 3356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:28:49.041612 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:28:49.041629 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:28:49.042091 (+ 462us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:28:49.042493 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:28:49.042498 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:28:49.042502 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:28:49.042713 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:28:49.043277 (+ 564us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:28:49.043286 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:28:49.043634 (+ 348us) client_negotiation.cc:769] Sending connection context
0806 09:28:49.043910 (+ 276us) client_negotiation.cc:240] Negotiation successful
0806 09:28:49.044507 (+ 597us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":288,"threads_started":1}
I20260806 09:28:49.047608 26062 heartbeater.cc:347] Connected to a master server at 127.20.113.62:37765
I20260806 09:28:49.047907 26062 heartbeater.cc:464] Registering TS with master...
I20260806 09:28:49.048714 26062 heartbeater.cc:511] Master 127.20.113.62:37765 requested a full tablet report, sending...
I20260806 09:28:49.050191 25422 ts_manager.cc:194] Registered new tserver with Master: afeb1420748548cb8c1ee20198d5542d (127.20.113.3:42463)
I20260806 09:28:49.051039 25422 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.113.3@KRBTEST.COM'} at 127.20.113.3:41347
I20260806 09:28:49.059724 20932 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 09:28:49 dist-test-slave-45dx krb5kdc[25360](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008529, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 06 09:28:49 dist-test-slave-45dx krb5kdc[25360](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008529, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.20.113.62@KRBTEST.COM
I20260806 09:28:49.118875 25874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:49.105452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57186 (local address 127.20.113.62:37765)
0806 09:28:49.105638 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:49.105643 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:49.106144 (+ 501us) server_negotiation.cc:407] Connection header received
0806 09:28:49.106298 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:49.106302 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:49.106391 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:49.106480 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:28:49.107785 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:49.108659 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:49.109803 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:49.110200 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:49.110289 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:49.114025 (+ 3736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:28:49.114030 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:28:49.114033 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:28:49.114065 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:28:49.116768 (+ 2703us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:49.117290 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:49.117293 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:49.117295 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:49.117421 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:49.117722 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:49.117725 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:49.117726 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:49.118020 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:28:49.118229 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:49.118454 (+ 225us) server_negotiation.cc:299] Negotiation successful
0806 09:28:49.118611 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
W20260806 09:28:49.119509 25422 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:57186
I20260806 09:28:49.121835 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 25510
Aug 06 09:28:49 dist-test-slave-45dx krb5kdc[25360](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008529, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.20.113.1@KRBTEST.COM
I20260806 09:28:49.137952 26077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:49.122808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60964 (local address 127.20.113.1:40341)
0806 09:28:49.123466 (+ 658us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:49.123473 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:49.123517 (+ 44us) server_negotiation.cc:407] Connection header received
0806 09:28:49.123572 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:49.123578 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:49.123783 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:49.123910 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:28:49.125064 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:49.126060 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:49.127191 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:49.127765 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:49.127865 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:49.131827 (+ 3962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:28:49.131840 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:28:49.131854 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:28:49.131903 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:28:49.135046 (+ 3143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:49.135757 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:49.135765 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:49.135767 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:49.135893 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:49.136278 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:49.136283 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:49.136286 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:49.136706 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:28:49.137027 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:49.137316 (+ 289us) server_negotiation.cc:299] Negotiation successful
0806 09:28:49.137599 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":498,"thread_start_us":323,"threads_started":1}
W20260806 09:28:49.139217 25679 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:60964
W20260806 09:28:49.140398 20932 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 25510: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 09:28:49.140587 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 25510
I20260806 09:28:49.165459 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 25698
Aug 06 09:28:49 dist-test-slave-45dx krb5kdc[25360](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008529, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.20.113.2@KRBTEST.COM
I20260806 09:28:49.181722 26079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:49.166409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38264 (local address 127.20.113.2:41775)
0806 09:28:49.166956 (+ 547us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:49.166963 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:49.166989 (+ 26us) server_negotiation.cc:407] Connection header received
0806 09:28:49.167051 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:49.167057 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:49.167269 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:49.167410 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:28:49.168719 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:49.169750 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:49.171036 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:49.171605 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:49.171705 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:49.175981 (+ 4276us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:28:49.175994 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:28:49.176009 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:28:49.176043 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:28:49.179121 (+ 3078us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:49.179755 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:49.179762 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:49.179764 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:49.179895 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:49.180228 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:49.180231 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:49.180233 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:49.180654 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:28:49.180900 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:49.181192 (+ 292us) server_negotiation.cc:299] Negotiation successful
0806 09:28:49.181420 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":282,"threads_started":1}
W20260806 09:28:49.183146 25861 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:38264
W20260806 09:28:49.184381 20932 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 25698: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 09:28:49.184564 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 25698
I20260806 09:28:49.211606 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 25880
Aug 06 09:28:49 dist-test-slave-45dx krb5kdc[25360](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008529, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.20.113.3@KRBTEST.COM
I20260806 09:28:49.228541 26083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:49.212785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46876 (local address 127.20.113.3:42463)
0806 09:28:49.213469 (+ 684us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:49.213476 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:49.213511 (+ 35us) server_negotiation.cc:407] Connection header received
0806 09:28:49.213572 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:49.213578 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:49.213784 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:49.213933 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:28:49.215229 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:49.216248 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:49.217577 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:49.218145 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:49.218241 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:49.222764 (+ 4523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:28:49.222777 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:28:49.222792 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:28:49.222826 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:28:49.225909 (+ 3083us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:49.226583 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:49.226591 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:49.226593 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:49.226714 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:49.227039 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:49.227042 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:49.227044 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:49.227372 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:28:49.227606 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:49.227858 (+ 252us) server_negotiation.cc:299] Negotiation successful
0806 09:28:49.228077 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":539,"thread_start_us":344,"threads_started":1}
W20260806 09:28:49.229905 26055 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:46876
W20260806 09:28:49.231009 20932 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 25880: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 09:28:49.231192 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 25880
I20260806 09:28:49.253713 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 25377
I20260806 09:28:49.254972 25466 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:28:49.359370 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 25377
2026-08-06T09:28:49Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (3943 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786008495763054-20932-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 09:28:49 dist-test-slave-45dx krb5kdc[26089](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 09:28:49 dist-test-slave-45dx krb5kdc[26089](info): set up 2 sockets
Aug 06 09:28:49 dist-test-slave-45dx krb5kdc[26089](info): commencing operation
krb5kdc: starting...
W20260806 09:28:50.657613 20932 mini_kdc.cc:121] Time spent starting KDC: real 1.246s user 0.000s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786008495763054-20932-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786008495763054-20932-0/krb5kdc/test-user.keytab.
Aug 06 09:28:50 dist-test-slave-45dx krb5kdc[26089](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008530, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T09:28:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T09:28:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.113.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.113.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.113.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab.
Entry for principal kudu/127.20.113.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab.
I20260806 09:28:50.896003 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786008495763054-20932-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786008495763054-20932-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786008495763054-20932-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.113.62:43763
--webserver_interface=127.20.113.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab
--principal=kudu/127.20.113.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:37433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.113.62:43763
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:28:51.232385 26105 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:28:51.232717 26105 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:28:51.232904 26105 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:28:51.239548 26105 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 09:28:51.239683 26105 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:28:51.239789 26105 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:28:51.239883 26105 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 09:28:51.239948 26105 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 09:28:51.249274 26105 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:37433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786008495763054-20932-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786008495763054-20932-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.113.62:43763
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab
--principal=kudu/127.20.113.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.113.62:43763
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.113.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786008495763054-20932-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:28:51.250589 26105 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:28:51.251940 26105 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:28:51.264025 26111 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:28:51.263940 26113 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:28:51.264050 26118 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:28:51.264770 26105 server_base.cc:1034] running on GCE node
I20260806 09:28:51.265748 26105 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:28:51.267113 26105 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:28:51.268424 26105 hybrid_clock.cc:648] HybridClock initialized: now 1786008531268369 us; error 27 us; skew 500 ppm
I20260806 09:28:51.268738 26105 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:28:51 dist-test-slave-45dx krb5kdc[26089](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008531, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:28:51.271869 26105 init.cc:377] Logged in from keytab as kudu/127.20.113.62@KRBTEST.COM (short username kudu)
I20260806 09:28:51.273372 26105 webserver.cc:460] Webserver started at http://127.20.113.62:35565/ using document root <none> and password file <none>
I20260806 09:28:51.274082 26105 fs_manager.cc:359] Metadata directory not provided
I20260806 09:28:51.274220 26105 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:28:51.274565 26105 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:28:51.277632 26105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786008495763054-20932-0/minicluster-data/master-0/data/instance:
uuid: "bf2b85d27f1b4b25a996b89e33016755"
format_stamp: "Formatted at 2026-08-06 09:28:51 on dist-test-slave-45dx"
server_key: "4ee07224c5082ac093b5c2a6fd4ab627"
server_key_iv: "568b582d080f962a8091c7da4a33be39"
server_key_version: "encryptionkey@0"
I20260806 09:28:51.278443 26105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786008495763054-20932-0/minicluster-data/master-0/wal/instance:
uuid: "bf2b85d27f1b4b25a996b89e33016755"
format_stamp: "Formatted at 2026-08-06 09:28:51 on dist-test-slave-45dx"
server_key: "4ee07224c5082ac093b5c2a6fd4ab627"
server_key_iv: "568b582d080f962a8091c7da4a33be39"
server_key_version: "encryptionkey@0"
I20260806 09:28:51.283439 26105 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 09:28:51.286903 26127 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:28:51.288201 26105 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 09:28:51.288398 26105 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786008495763054-20932-0/minicluster-data/master-0/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786008495763054-20932-0/minicluster-data/master-0/wal
uuid: "bf2b85d27f1b4b25a996b89e33016755"
format_stamp: "Formatted at 2026-08-06 09:28:51 on dist-test-slave-45dx"
server_key: "4ee07224c5082ac093b5c2a6fd4ab627"
server_key_iv: "568b582d080f962a8091c7da4a33be39"
server_key_version: "encryptionkey@0"
I20260806 09:28:51.288587 26105 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786008495763054-20932-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786008495763054-20932-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786008495763054-20932-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:28:51.304844 26105 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:28:51.309106 26105 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:28:51.309393 26105 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:28:51.328665 26105 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.62:43763
I20260806 09:28:51.328702 26201 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.62:43763 every 8 connection(s)
I20260806 09:28:51.330099 26105 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
I20260806 09:28:51.333914 26202 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:28:51.337478 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 26105
I20260806 09:28:51.337724 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786008495763054-20932-0/minicluster-data/master-0/wal/instance
I20260806 09:28:51.338593 20932 external_mini_cluster.cc:1428] Setting key 64ca580eef2200eab99fe88cd7609c0d
I20260806 09:28:51.347584 26202 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf2b85d27f1b4b25a996b89e33016755: Bootstrap starting.
I20260806 09:28:51.351455 26202 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bf2b85d27f1b4b25a996b89e33016755: Neither blocks nor log segments found. Creating new log.
I20260806 09:28:51.352824 26202 log.cc:824] T 00000000000000000000000000000000 P bf2b85d27f1b4b25a996b89e33016755: Log is configured to *not* fsync() on all Append() calls
Aug 06 09:28:51 dist-test-slave-45dx krb5kdc[26089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008530, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.113.62@KRBTEST.COM
I20260806 09:28:51.355962 26202 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf2b85d27f1b4b25a996b89e33016755: No bootstrap required, opened a new log
I20260806 09:28:51.362900 26205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:51.340846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48300 (local address 127.20.113.62:43763)
0806 09:28:51.341638 (+ 792us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:51.341649 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:51.341690 (+ 41us) server_negotiation.cc:407] Connection header received
0806 09:28:51.343398 (+ 1708us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:51.343421 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:51.343833 (+ 412us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:51.344152 (+ 319us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:28:51.346078 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:51.347383 (+ 1305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:51.349125 (+ 1742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:51.349817 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:51.349945 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:51.355154 (+ 5209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:28:51.355161 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:28:51.355174 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:28:51.355211 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:28:51.358595 (+ 3384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:51.359469 (+ 874us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:51.359474 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:51.359480 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:51.359643 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:51.360044 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:51.360047 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:51.360049 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:51.360665 (+ 616us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:28:51.360914 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:51.361389 (+ 475us) server_negotiation.cc:299] Negotiation successful
0806 09:28:51.361813 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":591,"thread_start_us":380,"threads_started":1}
I20260806 09:28:51.365267 26202 raft_consensus.cc:348] T 00000000000000000000000000000000 P bf2b85d27f1b4b25a996b89e33016755 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf2b85d27f1b4b25a996b89e33016755" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 43763 } }
I20260806 09:28:51.365679 26202 raft_consensus.cc:374] T 00000000000000000000000000000000 P bf2b85d27f1b4b25a996b89e33016755 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:28:51.365810 26202 raft_consensus.cc:729] T 00000000000000000000000000000000 P bf2b85d27f1b4b25a996b89e33016755 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf2b85d27f1b4b25a996b89e33016755, State: Initialized, Role: FOLLOWER
I20260806 09:28:51.366423 26202 consensus_queue.cc:260] T 00000000000000000000000000000000 P bf2b85d27f1b4b25a996b89e33016755 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf2b85d27f1b4b25a996b89e33016755" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 43763 } }
I20260806 09:28:51.366660 26202 raft_consensus.cc:388] T 00000000000000000000000000000000 P bf2b85d27f1b4b25a996b89e33016755 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 09:28:51.366796 26202 raft_consensus.cc:482] T 00000000000000000000000000000000 P bf2b85d27f1b4b25a996b89e33016755 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 09:28:51.366969 26202 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bf2b85d27f1b4b25a996b89e33016755 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:28:51.369236 26202 raft_consensus.cc:504] T 00000000000000000000000000000000 P bf2b85d27f1b4b25a996b89e33016755 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf2b85d27f1b4b25a996b89e33016755" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 43763 } }
I20260806 09:28:51.369745 26202 leader_election.cc:302] T 00000000000000000000000000000000 P bf2b85d27f1b4b25a996b89e33016755 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bf2b85d27f1b4b25a996b89e33016755; no voters:
I20260806 09:28:51.370463 26202 leader_election.cc:288] T 00000000000000000000000000000000 P bf2b85d27f1b4b25a996b89e33016755 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 09:28:51.370582 26209 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bf2b85d27f1b4b25a996b89e33016755 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:28:51.371423 26209 raft_consensus.cc:686] T 00000000000000000000000000000000 P bf2b85d27f1b4b25a996b89e33016755 [term 1 LEADER]: Becoming Leader. State: Replica: bf2b85d27f1b4b25a996b89e33016755, State: Running, Role: LEADER
I20260806 09:28:51.371924 26209 consensus_queue.cc:237] T 00000000000000000000000000000000 P bf2b85d27f1b4b25a996b89e33016755 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf2b85d27f1b4b25a996b89e33016755" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 43763 } }
I20260806 09:28:51.372656 26202 sys_catalog.cc:564] T 00000000000000000000000000000000 P bf2b85d27f1b4b25a996b89e33016755 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 09:28:51.374936 26210 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf2b85d27f1b4b25a996b89e33016755 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bf2b85d27f1b4b25a996b89e33016755" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf2b85d27f1b4b25a996b89e33016755" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 43763 } } }
I20260806 09:28:51.375419 26210 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf2b85d27f1b4b25a996b89e33016755 [sys.catalog]: This master's current role is: LEADER
I20260806 09:28:51.374969 26211 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf2b85d27f1b4b25a996b89e33016755 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bf2b85d27f1b4b25a996b89e33016755. Latest consensus state: current_term: 1 leader_uuid: "bf2b85d27f1b4b25a996b89e33016755" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf2b85d27f1b4b25a996b89e33016755" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 43763 } } }
I20260806 09:28:51.376902 26211 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf2b85d27f1b4b25a996b89e33016755 [sys.catalog]: This master's current role is: LEADER
I20260806 09:28:51.378738 26219 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 09:28:51.384027 26219 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 09:28:51.392642 26219 catalog_manager.cc:1347] Generated new cluster ID: 7a5acabb194a41abb6498ea4fa256c62
I20260806 09:28:51.392769 26219 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 09:28:51.401717 26219 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 09:28:51.402895 26219 catalog_manager.cc:1504] Loading token signing keys...
I20260806 09:28:51.414553 26219 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bf2b85d27f1b4b25a996b89e33016755: Generated new TSK 0
I20260806 09:28:51.415320 26219 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.113.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.113.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.113.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.1.keytab.
Entry for principal kudu/127.20.113.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.1.keytab.
I20260806 09:28:51.479789 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786008495763054-20932-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786008495763054-20932-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786008495763054-20932-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786008495763054-20932-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.113.1:0
--local_ip_for_outbound_sockets=127.20.113.1
--webserver_interface=127.20.113.1
--webserver_port=0
--tserver_master_addrs=127.20.113.62:43763
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.1.keytab
--principal=kudu/127.20.113.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:37433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:28:51.823082 26238 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:28:51.823359 26238 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:28:51.823553 26238 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:28:51.830543 26238 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:28:51.830662 26238 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:28:51.830840 26238 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.113.1
I20260806 09:28:51.840114 26238 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:37433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786008495763054-20932-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786008495763054-20932-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.1.keytab
--principal=kudu/127.20.113.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.113.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786008495763054-20932-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.113.1
--webserver_port=0
--tserver_master_addrs=127.20.113.62:43763
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.113.1
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786008495763054-20932-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:28:51.841396 26238 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:28:51.842710 26238 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:28:51.856129 26246 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:28:51.856159 26247 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:28:51.856062 26250 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:28:51.856837 26238 server_base.cc:1034] running on GCE node
I20260806 09:28:51.857447 26238 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:28:51.858392 26238 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:28:51.859602 26238 hybrid_clock.cc:648] HybridClock initialized: now 1786008531859553 us; error 44 us; skew 500 ppm
I20260806 09:28:51.859952 26238 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:28:51 dist-test-slave-45dx krb5kdc[26089](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008531, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:28:51.863248 26238 init.cc:377] Logged in from keytab as kudu/127.20.113.1@KRBTEST.COM (short username kudu)
I20260806 09:28:51.864936 26238 webserver.cc:460] Webserver started at http://127.20.113.1:41917/ using document root <none> and password file <none>
I20260806 09:28:51.865607 26238 fs_manager.cc:359] Metadata directory not provided
I20260806 09:28:51.865761 26238 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:28:51.866098 26238 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:28:51.869163 26238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786008495763054-20932-0/minicluster-data/ts-0/data/instance:
uuid: "120e0a1a9ace449ea2d95ef8e515174d"
format_stamp: "Formatted at 2026-08-06 09:28:51 on dist-test-slave-45dx"
server_key: "050006eb5bdea647a5070b35ed63d6ba"
server_key_iv: "d781910486b849ed80ee0a5023e0986a"
server_key_version: "encryptionkey@0"
I20260806 09:28:51.869942 26238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786008495763054-20932-0/minicluster-data/ts-0/wal/instance:
uuid: "120e0a1a9ace449ea2d95ef8e515174d"
format_stamp: "Formatted at 2026-08-06 09:28:51 on dist-test-slave-45dx"
server_key: "050006eb5bdea647a5070b35ed63d6ba"
server_key_iv: "d781910486b849ed80ee0a5023e0986a"
server_key_version: "encryptionkey@0"
I20260806 09:28:51.875306 26238 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 09:28:51.879081 26257 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:28:51.880393 26238 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 09:28:51.880551 26238 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786008495763054-20932-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786008495763054-20932-0/minicluster-data/ts-0/wal
uuid: "120e0a1a9ace449ea2d95ef8e515174d"
format_stamp: "Formatted at 2026-08-06 09:28:51 on dist-test-slave-45dx"
server_key: "050006eb5bdea647a5070b35ed63d6ba"
server_key_iv: "d781910486b849ed80ee0a5023e0986a"
server_key_version: "encryptionkey@0"
I20260806 09:28:51.880761 26238 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786008495763054-20932-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786008495763054-20932-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786008495763054-20932-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:28:51.897099 26238 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:28:51.900934 26238 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:28:51.901239 26238 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:28:51.902154 26238 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:28:51.903445 26238 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:28:51.903518 26238 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:28:51.903638 26238 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:28:51.903697 26238 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:28:51.937686 26238 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.1:41757
I20260806 09:28:51.937732 26417 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.1:41757 every 8 connection(s)
I20260806 09:28:51.939035 26238 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786008495763054-20932-0/minicluster-data/ts-0/data/info.pb
I20260806 09:28:51.946086 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 26238
I20260806 09:28:51.946312 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786008495763054-20932-0/minicluster-data/ts-0/wal/instance
I20260806 09:28:51.947173 20932 external_mini_cluster.cc:1428] Setting key 2f2a2cc171f48c6d8f2d211fc749fc90
Aug 06 09:28:51 dist-test-slave-45dx krb5kdc[26089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008531, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.1@KRBTEST.COM for kudu/127.20.113.62@KRBTEST.COM
I20260806 09:28:51.961334 26421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:51.941554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.1:33021 (local address 127.20.113.62:43763)
0806 09:28:51.942037 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:51.942042 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:51.942857 (+ 815us) server_negotiation.cc:407] Connection header received
0806 09:28:51.944725 (+ 1868us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:51.944731 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:51.944823 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:51.944947 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:28:51.946533 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:51.947654 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:51.950016 (+ 2362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:51.950689 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:51.950828 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:51.954501 (+ 3673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:28:51.954510 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:28:51.954517 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:28:51.954562 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:28:51.958169 (+ 3607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:51.958887 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:51.958895 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:51.958901 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:51.959052 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:51.959449 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:51.959456 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:51.959461 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:51.959848 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:28:51.960015 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:51.960900 (+ 885us) server_negotiation.cc:299] Negotiation successful
0806 09:28:51.961070 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":287,"threads_started":1}
I20260806 09:28:51.962853 26423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:51.941973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:43763 (local address 127.20.113.1:33021)
0806 09:28:51.942679 (+ 706us) negotiation.cc:107] Waiting for socket to connect
0806 09:28:51.942724 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 09:28:51.944458 (+ 1734us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:28:51.945182 (+ 724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:28:51.945190 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:28:51.945718 (+ 528us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:28:51.946324 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:51.946343 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:51.947867 (+ 1524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:51.947872 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:51.949814 (+ 1942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:51.949830 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:51.950934 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:51.950943 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:51.951259 (+ 316us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:51.952025 (+ 766us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:28:51.952048 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:28:51.954270 (+ 2222us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:28:51.958310 (+ 4040us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:28:51.958316 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:28:51.958329 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:28:51.958738 (+ 409us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:28:51.959156 (+ 418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:28:51.959159 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:28:51.959161 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:28:51.959334 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:28:51.959981 (+ 647us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:28:51.959987 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:28:51.960459 (+ 472us) client_negotiation.cc:769] Sending connection context
0806 09:28:51.960786 (+ 327us) client_negotiation.cc:240] Negotiation successful
0806 09:28:51.961360 (+ 574us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":286,"threads_started":1}
I20260806 09:28:51.964427 26418 heartbeater.cc:347] Connected to a master server at 127.20.113.62:43763
I20260806 09:28:51.964732 26418 heartbeater.cc:464] Registering TS with master...
I20260806 09:28:51.965474 26418 heartbeater.cc:511] Master 127.20.113.62:43763 requested a full tablet report, sending...
I20260806 09:28:51.967567 26149 ts_manager.cc:194] Registered new tserver with Master: 120e0a1a9ace449ea2d95ef8e515174d (127.20.113.1:41757)
I20260806 09:28:51.969012 26149 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.113.1@KRBTEST.COM'} at 127.20.113.1:33021
WARNING: no policy specified for kudu/127.20.113.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.113.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.113.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.2.keytab.
Entry for principal kudu/127.20.113.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.2.keytab.
I20260806 09:28:52.005134 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786008495763054-20932-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786008495763054-20932-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786008495763054-20932-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786008495763054-20932-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.113.2:0
--local_ip_for_outbound_sockets=127.20.113.2
--webserver_interface=127.20.113.2
--webserver_port=0
--tserver_master_addrs=127.20.113.62:43763
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.2.keytab
--principal=kudu/127.20.113.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:37433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:28:52.353302 26427 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:28:52.353605 26427 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:28:52.353775 26427 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:28:52.361181 26427 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:28:52.361346 26427 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:28:52.361526 26427 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.113.2
I20260806 09:28:52.371400 26427 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:37433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786008495763054-20932-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786008495763054-20932-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.2.keytab
--principal=kudu/127.20.113.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.113.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786008495763054-20932-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.113.2
--webserver_port=0
--tserver_master_addrs=127.20.113.62:43763
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.113.2
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786008495763054-20932-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:28:52.372923 26427 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:28:52.374404 26427 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:28:52.388319 26433 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:28:52.388352 26436 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:28:52.388392 26434 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:28:52.389272 26427 server_base.cc:1034] running on GCE node
I20260806 09:28:52.389882 26427 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:28:52.390753 26427 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:28:52.391966 26427 hybrid_clock.cc:648] HybridClock initialized: now 1786008532391911 us; error 38 us; skew 500 ppm
I20260806 09:28:52.392287 26427 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:28:52 dist-test-slave-45dx krb5kdc[26089](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008532, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:28:52.395578 26427 init.cc:377] Logged in from keytab as kudu/127.20.113.2@KRBTEST.COM (short username kudu)
I20260806 09:28:52.397101 26427 webserver.cc:460] Webserver started at http://127.20.113.2:45583/ using document root <none> and password file <none>
I20260806 09:28:52.397800 26427 fs_manager.cc:359] Metadata directory not provided
I20260806 09:28:52.397957 26427 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:28:52.398288 26427 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:28:52.401378 26427 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786008495763054-20932-0/minicluster-data/ts-1/data/instance:
uuid: "96bb11d32c3a4970a4b983b23ec90236"
format_stamp: "Formatted at 2026-08-06 09:28:52 on dist-test-slave-45dx"
server_key: "2d16996f8a73be645d5d214eca79e527"
server_key_iv: "69b1dfa4f6de98f0b06800e01bdb9887"
server_key_version: "encryptionkey@0"
I20260806 09:28:52.402189 26427 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786008495763054-20932-0/minicluster-data/ts-1/wal/instance:
uuid: "96bb11d32c3a4970a4b983b23ec90236"
format_stamp: "Formatted at 2026-08-06 09:28:52 on dist-test-slave-45dx"
server_key: "2d16996f8a73be645d5d214eca79e527"
server_key_iv: "69b1dfa4f6de98f0b06800e01bdb9887"
server_key_version: "encryptionkey@0"
I20260806 09:28:52.407433 26427 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260806 09:28:52.411554 26443 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:28:52.413010 26427 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 09:28:52.413188 26427 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786008495763054-20932-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786008495763054-20932-0/minicluster-data/ts-1/wal
uuid: "96bb11d32c3a4970a4b983b23ec90236"
format_stamp: "Formatted at 2026-08-06 09:28:52 on dist-test-slave-45dx"
server_key: "2d16996f8a73be645d5d214eca79e527"
server_key_iv: "69b1dfa4f6de98f0b06800e01bdb9887"
server_key_version: "encryptionkey@0"
I20260806 09:28:52.413400 26427 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786008495763054-20932-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786008495763054-20932-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786008495763054-20932-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:28:52.426795 26427 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:28:52.430828 26427 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:28:52.431147 26427 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:28:52.432142 26427 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:28:52.433573 26427 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:28:52.433678 26427 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:28:52.433828 26427 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:28:52.433907 26427 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:28:52.468421 26427 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.2:44383
I20260806 09:28:52.468474 26622 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.2:44383 every 8 connection(s)
I20260806 09:28:52.469642 26427 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786008495763054-20932-0/minicluster-data/ts-1/data/info.pb
I20260806 09:28:52.470587 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 26427
I20260806 09:28:52.470801 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786008495763054-20932-0/minicluster-data/ts-1/wal/instance
I20260806 09:28:52.471385 20932 external_mini_cluster.cc:1428] Setting key 073cb345a059944e77770b64e053cf0d
Aug 06 09:28:52 dist-test-slave-45dx krb5kdc[26089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008532, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.2@KRBTEST.COM for kudu/127.20.113.62@KRBTEST.COM
I20260806 09:28:52.491477 26629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:52.472473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.2:33699 (local address 127.20.113.62:43763)
0806 09:28:52.472983 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:52.472988 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:52.474253 (+ 1265us) server_negotiation.cc:407] Connection header received
0806 09:28:52.476027 (+ 1774us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:52.476032 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:52.476113 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:52.476238 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:28:52.477824 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:52.478717 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:52.480299 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:52.480766 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:52.480915 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:52.484471 (+ 3556us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:28:52.484475 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:28:52.484479 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:28:52.484512 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:28:52.488042 (+ 3530us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:52.488849 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:52.488857 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:52.488862 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:52.489023 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:52.489457 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:52.489464 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:52.489470 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:52.489824 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:28:52.490002 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:52.490652 (+ 650us) server_negotiation.cc:299] Negotiation successful
0806 09:28:52.490948 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":401,"thread_start_us":267,"threads_started":1}
I20260806 09:28:52.492048 26631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:52.473209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:43763 (local address 127.20.113.2:33699)
0806 09:28:52.474111 (+ 902us) negotiation.cc:107] Waiting for socket to connect
0806 09:28:52.474155 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 09:28:52.475766 (+ 1611us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:28:52.476491 (+ 725us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:28:52.476499 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:28:52.477002 (+ 503us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:28:52.477662 (+ 660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:52.477680 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:52.478914 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:52.478919 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:52.480148 (+ 1229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:52.480160 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:52.481023 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:52.481031 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:52.481261 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:52.481973 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:28:52.481996 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:28:52.484198 (+ 2202us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:28:52.488195 (+ 3997us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:28:52.488203 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:28:52.488214 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:28:52.488699 (+ 485us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:28:52.489137 (+ 438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:28:52.489140 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:28:52.489142 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:28:52.489338 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:28:52.489960 (+ 622us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:28:52.489966 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:28:52.490283 (+ 317us) client_negotiation.cc:769] Sending connection context
0806 09:28:52.490505 (+ 222us) client_negotiation.cc:240] Negotiation successful
0806 09:28:52.490902 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":604,"thread_start_us":317,"threads_started":1}
I20260806 09:28:52.493511 26625 heartbeater.cc:347] Connected to a master server at 127.20.113.62:43763
I20260806 09:28:52.493822 26625 heartbeater.cc:464] Registering TS with master...
I20260806 09:28:52.494819 26625 heartbeater.cc:511] Master 127.20.113.62:43763 requested a full tablet report, sending...
I20260806 09:28:52.496505 26149 ts_manager.cc:194] Registered new tserver with Master: 96bb11d32c3a4970a4b983b23ec90236 (127.20.113.2:44383)
I20260806 09:28:52.497315 26149 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.113.2@KRBTEST.COM'} at 127.20.113.2:33699
WARNING: no policy specified for kudu/127.20.113.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.113.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.113.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.3.keytab.
Entry for principal kudu/127.20.113.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.3.keytab.
I20260806 09:28:52.535905 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786008495763054-20932-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786008495763054-20932-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786008495763054-20932-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786008495763054-20932-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.113.3:0
--local_ip_for_outbound_sockets=127.20.113.3
--webserver_interface=127.20.113.3
--webserver_port=0
--tserver_master_addrs=127.20.113.62:43763
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.3.keytab
--principal=kudu/127.20.113.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:37433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:28:52.880333 26635 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:28:52.880607 26635 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:28:52.880702 26635 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:28:52.887769 26635 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:28:52.887866 26635 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:28:52.887969 26635 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.113.3
I20260806 09:28:52.897552 26635 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:37433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786008495763054-20932-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786008495763054-20932-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.3.keytab
--principal=kudu/127.20.113.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.113.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786008495763054-20932-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.113.3
--webserver_port=0
--tserver_master_addrs=127.20.113.62:43763
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.113.3
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786008495763054-20932-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:28:52.898727 26635 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:28:52.899989 26635 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:28:52.913250 26648 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:28:52.913424 26643 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:28:52.913457 26642 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:28:52.914402 26635 server_base.cc:1034] running on GCE node
I20260806 09:28:52.915067 26635 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:28:52.916064 26635 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:28:52.917309 26635 hybrid_clock.cc:648] HybridClock initialized: now 1786008532917217 us; error 80 us; skew 500 ppm
I20260806 09:28:52.917657 26635 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:28:52 dist-test-slave-45dx krb5kdc[26089](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008532, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:28:52.921123 26635 init.cc:377] Logged in from keytab as kudu/127.20.113.3@KRBTEST.COM (short username kudu)
I20260806 09:28:52.922663 26635 webserver.cc:460] Webserver started at http://127.20.113.3:34985/ using document root <none> and password file <none>
I20260806 09:28:52.923336 26635 fs_manager.cc:359] Metadata directory not provided
I20260806 09:28:52.923447 26635 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:28:52.923731 26635 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:28:52.926743 26635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786008495763054-20932-0/minicluster-data/ts-2/data/instance:
uuid: "01a94eec37424e2d9832c3d8b4a923eb"
format_stamp: "Formatted at 2026-08-06 09:28:52 on dist-test-slave-45dx"
server_key: "f17c00a97e45e9967a70920f070702c5"
server_key_iv: "40d5ff0518aa8fbe1fdf605a86075baa"
server_key_version: "encryptionkey@0"
I20260806 09:28:52.927541 26635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786008495763054-20932-0/minicluster-data/ts-2/wal/instance:
uuid: "01a94eec37424e2d9832c3d8b4a923eb"
format_stamp: "Formatted at 2026-08-06 09:28:52 on dist-test-slave-45dx"
server_key: "f17c00a97e45e9967a70920f070702c5"
server_key_iv: "40d5ff0518aa8fbe1fdf605a86075baa"
server_key_version: "encryptionkey@0"
I20260806 09:28:52.932920 26635 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.006s
I20260806 09:28:52.936394 26655 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:28:52.937943 26635 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 09:28:52.938138 26635 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786008495763054-20932-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786008495763054-20932-0/minicluster-data/ts-2/wal
uuid: "01a94eec37424e2d9832c3d8b4a923eb"
format_stamp: "Formatted at 2026-08-06 09:28:52 on dist-test-slave-45dx"
server_key: "f17c00a97e45e9967a70920f070702c5"
server_key_iv: "40d5ff0518aa8fbe1fdf605a86075baa"
server_key_version: "encryptionkey@0"
I20260806 09:28:52.938362 26635 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786008495763054-20932-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786008495763054-20932-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786008495763054-20932-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:28:52.953570 26635 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:28:52.958666 26635 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:28:52.958948 26635 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:28:52.959937 26635 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:28:52.961560 26635 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:28:52.961656 26635 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:28:52.961766 26635 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:28:52.961825 26635 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:28:52.971946 26418 heartbeater.cc:503] Master 127.20.113.62:43763 was elected leader, sending a full tablet report...
I20260806 09:28:53.000334 26635 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.3:44205
I20260806 09:28:53.000423 26819 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.3:44205 every 8 connection(s)
I20260806 09:28:53.001667 26635 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786008495763054-20932-0/minicluster-data/ts-2/data/info.pb
I20260806 09:28:53.002682 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 26635
I20260806 09:28:53.002902 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786008495763054-20932-0/minicluster-data/ts-2/wal/instance
I20260806 09:28:53.003497 20932 external_mini_cluster.cc:1428] Setting key db562a83546fc3bc505ab8252d2d28ef
Aug 06 09:28:53 dist-test-slave-45dx krb5kdc[26089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008532, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.3@KRBTEST.COM for kudu/127.20.113.62@KRBTEST.COM
I20260806 09:28:53.022364 26824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:53.004676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.3:46811 (local address 127.20.113.62:43763)
0806 09:28:53.005371 (+ 695us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:53.005377 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:53.006371 (+ 994us) server_negotiation.cc:407] Connection header received
0806 09:28:53.008277 (+ 1906us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:53.008283 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:53.008408 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:53.008528 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:28:53.010006 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:53.010787 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:53.012361 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:53.012770 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:53.012909 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:53.016268 (+ 3359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:28:53.016273 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:28:53.016276 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:28:53.016363 (+ 87us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:28:53.019703 (+ 3340us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:53.020421 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:53.020424 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:53.020426 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:53.020550 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:53.020952 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:53.020955 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:53.020957 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:53.021207 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:28:53.021319 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:53.021933 (+ 614us) server_negotiation.cc:299] Negotiation successful
0806 09:28:53.022083 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":554,"thread_start_us":809,"threads_started":1}
I20260806 09:28:53.023267 26825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:53.005344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:43763 (local address 127.20.113.3:46811)
0806 09:28:53.006224 (+ 880us) negotiation.cc:107] Waiting for socket to connect
0806 09:28:53.006262 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 09:28:53.008023 (+ 1761us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:28:53.008764 (+ 741us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:28:53.008773 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:28:53.009294 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:28:53.009831 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:53.009848 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:53.010961 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:53.010965 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:53.012175 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:53.012187 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:53.013004 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:53.013011 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:53.013231 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:53.013881 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:28:53.013904 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:28:53.016061 (+ 2157us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:28:53.019833 (+ 3772us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:28:53.019839 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:28:53.019863 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:28:53.020259 (+ 396us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:28:53.020658 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:28:53.020661 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:28:53.020663 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:28:53.020841 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:28:53.021323 (+ 482us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:28:53.021328 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:28:53.021610 (+ 282us) client_negotiation.cc:769] Sending connection context
0806 09:28:53.021839 (+ 229us) client_negotiation.cc:240] Negotiation successful
0806 09:28:53.022200 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":655,"thread_start_us":447,"threads_started":1}
I20260806 09:28:53.024657 26820 heartbeater.cc:347] Connected to a master server at 127.20.113.62:43763
I20260806 09:28:53.024964 26820 heartbeater.cc:464] Registering TS with master...
I20260806 09:28:53.025681 26820 heartbeater.cc:511] Master 127.20.113.62:43763 requested a full tablet report, sending...
I20260806 09:28:53.027068 26149 ts_manager.cc:194] Registered new tserver with Master: 01a94eec37424e2d9832c3d8b4a923eb (127.20.113.3:44205)
I20260806 09:28:53.027807 26149 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.113.3@KRBTEST.COM'} at 127.20.113.3:46811
I20260806 09:28:53.031466 20932 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 09:28:53 dist-test-slave-45dx krb5kdc[26089](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008533, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 09:28:53 dist-test-slave-45dx krb5kdc[26089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008533, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.113.1@KRBTEST.COM
I20260806 09:28:53.094854 26833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:53.078549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59122 (local address 127.20.113.1:41757)
0806 09:28:53.079257 (+ 708us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:53.079265 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:53.079296 (+ 31us) server_negotiation.cc:407] Connection header received
0806 09:28:53.079362 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:53.079369 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:53.079585 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:53.079726 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:28:53.081316 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:53.082426 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:53.083728 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:53.084459 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:53.084568 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:53.088967 (+ 4399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:28:53.088980 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:28:53.088995 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:28:53.089031 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:28:53.091973 (+ 2942us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:53.092815 (+ 842us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:53.092822 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:53.092824 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:53.092974 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:53.093379 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:53.093382 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:53.093384 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:53.093739 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:28:53.093974 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:53.094213 (+ 239us) server_negotiation.cc:299] Negotiation successful
0806 09:28:53.094430 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":552,"thread_start_us":386,"threads_started":1}
Aug 06 09:28:53 dist-test-slave-45dx krb5kdc[26089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008533, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.113.62@KRBTEST.COM
I20260806 09:28:53.113471 26824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:53.099983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43484 (local address 127.20.113.62:43763)
0806 09:28:53.100217 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:53.100223 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:53.100508 (+ 285us) server_negotiation.cc:407] Connection header received
0806 09:28:53.100706 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:53.100711 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:53.100796 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:53.100902 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:28:53.102258 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:53.103018 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:53.104232 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:53.104730 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:53.104868 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:53.108596 (+ 3728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:28:53.108600 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:28:53.108604 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:28:53.108635 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:28:53.111101 (+ 2466us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:53.111706 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:53.111710 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:53.111712 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:53.111830 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:53.112204 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:53.112207 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:53.112209 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:53.112570 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:28:53.112746 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:53.113077 (+ 331us) server_negotiation.cc:299] Negotiation successful
0806 09:28:53.113236 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
W20260806 09:28:53.114050 26149 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43484
I20260806 09:28:53.115051 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 26238
I20260806 09:28:53.127465 26833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:53.115942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59126 (local address 127.20.113.1:41757)
0806 09:28:53.116107 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:53.116111 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:53.116264 (+ 153us) server_negotiation.cc:407] Connection header received
0806 09:28:53.116452 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:53.116458 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:53.116554 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:53.116650 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:28:53.117958 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:53.118639 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:53.119757 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:53.120186 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:53.120283 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:53.122493 (+ 2210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:28:53.122498 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:28:53.122500 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:28:53.122530 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:28:53.125301 (+ 2771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:53.125902 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:53.125906 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:53.125908 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:53.126038 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:53.126377 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:53.126380 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:53.126382 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:53.126622 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:28:53.126813 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:53.127085 (+ 272us) server_negotiation.cc:299] Negotiation successful
0806 09:28:53.127242 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 09:28:53.128113 26412 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:28:53.211268 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 26238
I20260806 09:28:53.233829 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 26427
Aug 06 09:28:53 dist-test-slave-45dx krb5kdc[26089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008533, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.113.2@KRBTEST.COM
I20260806 09:28:53.250254 26840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:53.234735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41124 (local address 127.20.113.2:44383)
0806 09:28:53.235368 (+ 633us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:53.235376 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:53.235403 (+ 27us) server_negotiation.cc:407] Connection header received
0806 09:28:53.235466 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:53.235473 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:53.235700 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:53.235839 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:28:53.237120 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:53.238132 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:53.239347 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:53.239875 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:53.239968 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:53.244255 (+ 4287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:28:53.244267 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:28:53.244281 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:28:53.244364 (+ 83us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:28:53.247209 (+ 2845us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:53.247928 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:53.247935 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:53.247938 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:53.248078 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:53.248575 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:53.248582 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:53.248585 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:53.248966 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:28:53.249296 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:53.249619 (+ 323us) server_negotiation.cc:299] Negotiation successful
0806 09:28:53.249911 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":473,"thread_start_us":346,"threads_started":1}
I20260806 09:28:53.251576 26618 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:28:53.335698 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 26427
I20260806 09:28:53.369182 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 26635
Aug 06 09:28:53 dist-test-slave-45dx krb5kdc[26089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008533, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.113.3@KRBTEST.COM
I20260806 09:28:53.385967 26842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:53.370094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36154 (local address 127.20.113.3:44205)
0806 09:28:53.370777 (+ 683us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:53.370784 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:53.370815 (+ 31us) server_negotiation.cc:407] Connection header received
0806 09:28:53.370878 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:53.370885 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:53.371095 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:53.371225 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:28:53.372516 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:53.373712 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:53.374946 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:53.375508 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:53.375603 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:53.380169 (+ 4566us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:28:53.380180 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:28:53.380193 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:28:53.380225 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:28:53.383074 (+ 2849us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:53.383786 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:53.383793 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:53.383795 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:53.383920 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:53.384360 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:53.384364 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:53.384366 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:53.384690 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:28:53.385036 (+ 346us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:53.385343 (+ 307us) server_negotiation.cc:299] Negotiation successful
0806 09:28:53.385629 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":515,"thread_start_us":333,"threads_started":1}
I20260806 09:28:53.387215 26812 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:28:53.469877 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 26635
I20260806 09:28:53.494751 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 26105
I20260806 09:28:53.496008 26195 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:28:53.602165 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 26105
2026-08-06T09:28:53Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (4240 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786008495763054-20932-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 09:28:53 dist-test-slave-45dx krb5kdc[26847](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 09:28:53 dist-test-slave-45dx krb5kdc[26847](info): set up 2 sockets
Aug 06 09:28:53 dist-test-slave-45dx krb5kdc[26847](info): commencing operation
krb5kdc: starting...
W20260806 09:28:54.892418 20932 mini_kdc.cc:121] Time spent starting KDC: real 1.241s user 0.000s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786008495763054-20932-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786008495763054-20932-0/krb5kdc/test-user.keytab.
Aug 06 09:28:55 dist-test-slave-45dx krb5kdc[26847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008535, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T09:28:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T09:28:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.113.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.113.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.113.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab.
Entry for principal kudu/127.20.113.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab.
I20260806 09:28:55.136026 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786008495763054-20932-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786008495763054-20932-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786008495763054-20932-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.113.62:42681
--webserver_interface=127.20.113.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab
--principal=kudu/127.20.113.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:36669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.113.62:42681
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:28:55.467727 26862 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:28:55.468010 26862 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:28:55.468182 26862 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:28:55.475132 26862 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 09:28:55.475255 26862 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:28:55.475342 26862 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:28:55.475445 26862 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 09:28:55.475531 26862 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 09:28:55.485072 26862 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:36669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786008495763054-20932-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786008495763054-20932-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.113.62:42681
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab
--principal=kudu/127.20.113.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.113.62:42681
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.113.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786008495763054-20932-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:28:55.486353 26862 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:28:55.487728 26862 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:28:55.499742 26872 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:28:55.499868 26874 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:28:55.499980 26870 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:28:55.502709 26862 server_base.cc:1034] running on GCE node
I20260806 09:28:55.503491 26862 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:28:55.504961 26862 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:28:55.506160 26862 hybrid_clock.cc:648] HybridClock initialized: now 1786008535506126 us; error 45 us; skew 500 ppm
I20260806 09:28:55.506527 26862 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:28:55 dist-test-slave-45dx krb5kdc[26847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008535, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:28:55.509747 26862 init.cc:377] Logged in from keytab as kudu/127.20.113.62@KRBTEST.COM (short username kudu)
I20260806 09:28:55.510941 26862 fs_manager.cc:359] Metadata directory not provided
I20260806 09:28:55.511080 26862 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:28:55.511454 26862 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:28:55.514840 26862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786008495763054-20932-0/minicluster-data/master-0/data/instance:
uuid: "7eb19a968c444ac8983e47c29260c23f"
format_stamp: "Formatted at 2026-08-06 09:28:55 on dist-test-slave-45dx"
server_key: "f2d8686fa178c6ccdf939821e302447e"
server_key_iv: "c8edbc9b6724433eb08a0348257b0128"
server_key_version: "encryptionkey@0"
I20260806 09:28:55.515669 26862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786008495763054-20932-0/minicluster-data/master-0/wal/instance:
uuid: "7eb19a968c444ac8983e47c29260c23f"
format_stamp: "Formatted at 2026-08-06 09:28:55 on dist-test-slave-45dx"
server_key: "f2d8686fa178c6ccdf939821e302447e"
server_key_iv: "c8edbc9b6724433eb08a0348257b0128"
server_key_version: "encryptionkey@0"
I20260806 09:28:55.520700 26862 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260806 09:28:55.524066 26883 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:28:55.525271 26862 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 09:28:55.525446 26862 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786008495763054-20932-0/minicluster-data/master-0/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786008495763054-20932-0/minicluster-data/master-0/wal
uuid: "7eb19a968c444ac8983e47c29260c23f"
format_stamp: "Formatted at 2026-08-06 09:28:55 on dist-test-slave-45dx"
server_key: "f2d8686fa178c6ccdf939821e302447e"
server_key_iv: "c8edbc9b6724433eb08a0348257b0128"
server_key_version: "encryptionkey@0"
I20260806 09:28:55.525657 26862 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786008495763054-20932-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786008495763054-20932-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786008495763054-20932-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:28:55.561307 26862 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:28:55.565285 26862 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:28:55.565587 26862 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:28:55.584086 26862 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.62:42681
I20260806 09:28:55.584093 26955 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.62:42681 every 8 connection(s)
I20260806 09:28:55.584924 26862 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
I20260806 09:28:55.588531 26958 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:28:55.588718 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 26862
I20260806 09:28:55.588953 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786008495763054-20932-0/minicluster-data/master-0/wal/instance
I20260806 09:28:55.589811 20932 external_mini_cluster.cc:1428] Setting key d8f242458b52ece6f5b9b20bc9286e54
I20260806 09:28:55.602425 26958 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7eb19a968c444ac8983e47c29260c23f: Bootstrap starting.
Aug 06 09:28:55 dist-test-slave-45dx krb5kdc[26847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008535, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.113.62@KRBTEST.COM
I20260806 09:28:55.605980 26958 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7eb19a968c444ac8983e47c29260c23f: Neither blocks nor log segments found. Creating new log.
I20260806 09:28:55.607048 26958 log.cc:824] T 00000000000000000000000000000000 P 7eb19a968c444ac8983e47c29260c23f: Log is configured to *not* fsync() on all Append() calls
I20260806 09:28:55.609874 26958 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7eb19a968c444ac8983e47c29260c23f: No bootstrap required, opened a new log
I20260806 09:28:55.613020 26963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:55.591929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57746 (local address 127.20.113.62:42681)
0806 09:28:55.592761 (+ 832us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:55.592785 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:55.592828 (+ 43us) server_negotiation.cc:407] Connection header received
0806 09:28:55.594765 (+ 1937us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:55.594786 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:55.595128 (+ 342us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:55.595471 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:28:55.597303 (+ 1832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:55.598549 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:55.600256 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:55.600933 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:55.601055 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:55.605652 (+ 4597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:28:55.605660 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:28:55.605672 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:28:55.605708 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:28:55.609071 (+ 3363us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:55.609712 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:55.609722 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:55.609730 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:55.609899 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:55.610317 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:55.610320 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:55.610332 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:55.610815 (+ 483us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:28:55.611133 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:55.611603 (+ 470us) server_negotiation.cc:299] Negotiation successful
0806 09:28:55.612010 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":627,"thread_start_us":428,"threads_started":1}
I20260806 09:28:55.617796 26958 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7eb19a968c444ac8983e47c29260c23f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7eb19a968c444ac8983e47c29260c23f" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 42681 } }
I20260806 09:28:55.618192 26958 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7eb19a968c444ac8983e47c29260c23f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:28:55.618299 26958 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7eb19a968c444ac8983e47c29260c23f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7eb19a968c444ac8983e47c29260c23f, State: Initialized, Role: FOLLOWER
I20260806 09:28:55.618973 26958 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7eb19a968c444ac8983e47c29260c23f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7eb19a968c444ac8983e47c29260c23f" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 42681 } }
I20260806 09:28:55.619275 26958 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7eb19a968c444ac8983e47c29260c23f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 09:28:55.619402 26958 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7eb19a968c444ac8983e47c29260c23f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 09:28:55.619518 26958 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7eb19a968c444ac8983e47c29260c23f [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:28:55.621810 26958 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7eb19a968c444ac8983e47c29260c23f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7eb19a968c444ac8983e47c29260c23f" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 42681 } }
I20260806 09:28:55.622419 26958 leader_election.cc:302] T 00000000000000000000000000000000 P 7eb19a968c444ac8983e47c29260c23f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7eb19a968c444ac8983e47c29260c23f; no voters:
I20260806 09:28:55.623008 26958 leader_election.cc:288] T 00000000000000000000000000000000 P 7eb19a968c444ac8983e47c29260c23f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 09:28:55.623142 26967 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7eb19a968c444ac8983e47c29260c23f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:28:55.623888 26967 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7eb19a968c444ac8983e47c29260c23f [term 1 LEADER]: Becoming Leader. State: Replica: 7eb19a968c444ac8983e47c29260c23f, State: Running, Role: LEADER
I20260806 09:28:55.624504 26967 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7eb19a968c444ac8983e47c29260c23f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7eb19a968c444ac8983e47c29260c23f" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 42681 } }
I20260806 09:28:55.625314 26958 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7eb19a968c444ac8983e47c29260c23f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 09:28:55.627311 26970 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7eb19a968c444ac8983e47c29260c23f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7eb19a968c444ac8983e47c29260c23f. Latest consensus state: current_term: 1 leader_uuid: "7eb19a968c444ac8983e47c29260c23f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7eb19a968c444ac8983e47c29260c23f" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 42681 } } }
I20260806 09:28:55.627821 26970 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7eb19a968c444ac8983e47c29260c23f [sys.catalog]: This master's current role is: LEADER
I20260806 09:28:55.627434 26968 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7eb19a968c444ac8983e47c29260c23f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7eb19a968c444ac8983e47c29260c23f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7eb19a968c444ac8983e47c29260c23f" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 42681 } } }
I20260806 09:28:55.628114 26968 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7eb19a968c444ac8983e47c29260c23f [sys.catalog]: This master's current role is: LEADER
I20260806 09:28:55.629937 26978 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 09:28:55.634166 26978 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 09:28:55.642891 26978 catalog_manager.cc:1347] Generated new cluster ID: b20a20e14bc44ab0b87d9e59fc8b7583
I20260806 09:28:55.643002 26978 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 09:28:55.654680 26978 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 09:28:55.656433 26978 catalog_manager.cc:1504] Loading token signing keys...
I20260806 09:28:55.672574 26978 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7eb19a968c444ac8983e47c29260c23f: Generated new TSK 0
I20260806 09:28:55.673637 26978 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.113.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.113.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.113.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.1.keytab.
Entry for principal kudu/127.20.113.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.1.keytab.
I20260806 09:28:55.730082 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786008495763054-20932-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786008495763054-20932-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786008495763054-20932-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786008495763054-20932-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.113.1:0
--local_ip_for_outbound_sockets=127.20.113.1
--webserver_interface=127.20.113.1
--webserver_port=0
--tserver_master_addrs=127.20.113.62:42681
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.1.keytab
--principal=kudu/127.20.113.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:36669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:28:56.068920 26999 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:28:56.069226 26999 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:28:56.069402 26999 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:28:56.076207 26999 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:28:56.076398 26999 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:28:56.076575 26999 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.113.1
I20260806 09:28:56.085867 26999 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:36669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786008495763054-20932-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786008495763054-20932-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.1.keytab
--principal=kudu/127.20.113.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.113.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786008495763054-20932-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.113.1
--webserver_port=0
--tserver_master_addrs=127.20.113.62:42681
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.113.1
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786008495763054-20932-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:28:56.087105 26999 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:28:56.088433 26999 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:28:56.100831 27013 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:28:56.100919 27007 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:28:56.100890 27008 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:28:56.101996 26999 server_base.cc:1034] running on GCE node
I20260806 09:28:56.102628 26999 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:28:56.103603 26999 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:28:56.104820 26999 hybrid_clock.cc:648] HybridClock initialized: now 1786008536104790 us; error 47 us; skew 500 ppm
I20260806 09:28:56.105124 26999 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:28:56 dist-test-slave-45dx krb5kdc[26847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008536, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:28:56.108198 26999 init.cc:377] Logged in from keytab as kudu/127.20.113.1@KRBTEST.COM (short username kudu)
I20260806 09:28:56.109419 26999 fs_manager.cc:359] Metadata directory not provided
I20260806 09:28:56.109509 26999 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:28:56.109754 26999 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:28:56.112749 26999 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786008495763054-20932-0/minicluster-data/ts-0/data/instance:
uuid: "e2e792c28ea3454cbaa6dda6d86593f3"
format_stamp: "Formatted at 2026-08-06 09:28:56 on dist-test-slave-45dx"
server_key: "55d82ce3e00f3e7615d11728fef0dcaa"
server_key_iv: "aa18b16817c4158bf1572aa18a6756d4"
server_key_version: "encryptionkey@0"
I20260806 09:28:56.113519 26999 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786008495763054-20932-0/minicluster-data/ts-0/wal/instance:
uuid: "e2e792c28ea3454cbaa6dda6d86593f3"
format_stamp: "Formatted at 2026-08-06 09:28:56 on dist-test-slave-45dx"
server_key: "55d82ce3e00f3e7615d11728fef0dcaa"
server_key_iv: "aa18b16817c4158bf1572aa18a6756d4"
server_key_version: "encryptionkey@0"
I20260806 09:28:56.118259 26999 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 09:28:56.121567 27019 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:28:56.122786 26999 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 09:28:56.122967 26999 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786008495763054-20932-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786008495763054-20932-0/minicluster-data/ts-0/wal
uuid: "e2e792c28ea3454cbaa6dda6d86593f3"
format_stamp: "Formatted at 2026-08-06 09:28:56 on dist-test-slave-45dx"
server_key: "55d82ce3e00f3e7615d11728fef0dcaa"
server_key_iv: "aa18b16817c4158bf1572aa18a6756d4"
server_key_version: "encryptionkey@0"
I20260806 09:28:56.123232 26999 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786008495763054-20932-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786008495763054-20932-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786008495763054-20932-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:28:56.148595 26999 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:28:56.153110 26999 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:28:56.153409 26999 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:28:56.154423 26999 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:28:56.155817 26999 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:28:56.155949 26999 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:28:56.156080 26999 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:28:56.156185 26999 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:28:56.189774 26999 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.1:42057
I20260806 09:28:56.189862 27179 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.1:42057 every 8 connection(s)
I20260806 09:28:56.190627 26999 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786008495763054-20932-0/minicluster-data/ts-0/data/info.pb
I20260806 09:28:56.194232 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 26999
I20260806 09:28:56.194478 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786008495763054-20932-0/minicluster-data/ts-0/wal/instance
I20260806 09:28:56.195289 20932 external_mini_cluster.cc:1428] Setting key 7ff206c9ca25145c3ffb3d02d4daf680
Aug 06 09:28:56 dist-test-slave-45dx krb5kdc[26847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008536, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.1@KRBTEST.COM for kudu/127.20.113.62@KRBTEST.COM
I20260806 09:28:56.213320 27184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:56.193058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.1:39651 (local address 127.20.113.62:42681)
0806 09:28:56.193540 (+ 482us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:56.193544 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:56.194577 (+ 1033us) server_negotiation.cc:407] Connection header received
0806 09:28:56.196965 (+ 2388us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:56.196970 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:56.197047 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:56.197207 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:28:56.199223 (+ 2016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:56.199950 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:56.202448 (+ 2498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:56.202839 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:56.202947 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:56.207139 (+ 4192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:28:56.207144 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:28:56.207147 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:28:56.207178 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:28:56.209741 (+ 2563us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:56.210699 (+ 958us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:56.210703 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:56.210705 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:56.210820 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:56.211353 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:56.211357 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:56.211361 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:56.211743 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:28:56.211889 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:56.212859 (+ 970us) server_negotiation.cc:299] Negotiation successful
0806 09:28:56.213059 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":274,"threads_started":1}
I20260806 09:28:56.214841 27185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:56.193524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:42681 (local address 127.20.113.1:39651)
0806 09:28:56.194380 (+ 856us) negotiation.cc:107] Waiting for socket to connect
0806 09:28:56.194449 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0806 09:28:56.196706 (+ 2257us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:28:56.197463 (+ 757us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:28:56.197475 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:28:56.198223 (+ 748us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:28:56.199045 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:56.199073 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:56.200124 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:56.200133 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:56.202295 (+ 2162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:56.202313 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:56.203094 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:56.203104 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:56.203429 (+ 325us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:56.204179 (+ 750us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:28:56.204225 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:28:56.206922 (+ 2697us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:28:56.209890 (+ 2968us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:28:56.209901 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:28:56.209920 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:28:56.210583 (+ 663us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:28:56.210921 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:28:56.210927 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:28:56.210933 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:28:56.211218 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:28:56.211904 (+ 686us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:28:56.211916 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:28:56.212424 (+ 508us) client_negotiation.cc:769] Sending connection context
0806 09:28:56.212730 (+ 306us) client_negotiation.cc:240] Negotiation successful
0806 09:28:56.213344 (+ 614us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":546,"thread_start_us":335,"threads_started":1}
I20260806 09:28:56.216471 27181 heartbeater.cc:347] Connected to a master server at 127.20.113.62:42681
I20260806 09:28:56.216764 27181 heartbeater.cc:464] Registering TS with master...
I20260806 09:28:56.217391 27181 heartbeater.cc:511] Master 127.20.113.62:42681 requested a full tablet report, sending...
I20260806 09:28:56.219501 26909 ts_manager.cc:194] Registered new tserver with Master: e2e792c28ea3454cbaa6dda6d86593f3 (127.20.113.1:42057)
I20260806 09:28:56.221038 26909 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.113.1@KRBTEST.COM'} at 127.20.113.1:39651
WARNING: no policy specified for kudu/127.20.113.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.113.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.113.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.2.keytab.
Entry for principal kudu/127.20.113.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.2.keytab.
I20260806 09:28:56.253863 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786008495763054-20932-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786008495763054-20932-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786008495763054-20932-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786008495763054-20932-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.113.2:0
--local_ip_for_outbound_sockets=127.20.113.2
--webserver_interface=127.20.113.2
--webserver_port=0
--tserver_master_addrs=127.20.113.62:42681
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.2.keytab
--principal=kudu/127.20.113.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:36669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:28:56.598984 27190 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:28:56.599282 27190 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:28:56.599459 27190 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:28:56.606647 27190 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:28:56.606782 27190 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:28:56.606986 27190 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.113.2
I20260806 09:28:56.616627 27190 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:36669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786008495763054-20932-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786008495763054-20932-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.2.keytab
--principal=kudu/127.20.113.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.113.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786008495763054-20932-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.113.2
--webserver_port=0
--tserver_master_addrs=127.20.113.62:42681
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.113.2
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786008495763054-20932-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:28:56.617849 27190 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:28:56.619104 27190 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:28:56.632570 27195 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:28:56.632575 27196 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:28:56.632908 27199 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:28:56.634075 27190 server_base.cc:1034] running on GCE node
I20260806 09:28:56.634699 27190 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:28:56.635620 27190 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:28:56.636873 27190 hybrid_clock.cc:648] HybridClock initialized: now 1786008536636804 us; error 62 us; skew 500 ppm
I20260806 09:28:56.637190 27190 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:28:56 dist-test-slave-45dx krb5kdc[26847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008536, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:28:56.640409 27190 init.cc:377] Logged in from keytab as kudu/127.20.113.2@KRBTEST.COM (short username kudu)
I20260806 09:28:56.641593 27190 fs_manager.cc:359] Metadata directory not provided
I20260806 09:28:56.641729 27190 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:28:56.642086 27190 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:28:56.645152 27190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786008495763054-20932-0/minicluster-data/ts-1/data/instance:
uuid: "2b574edc4ab14a538db7020e110976cc"
format_stamp: "Formatted at 2026-08-06 09:28:56 on dist-test-slave-45dx"
server_key: "007ea35b25176759e3e0e4cbfa65a9ac"
server_key_iv: "fbc961d80f2d118e31eec3ca37c3d4a7"
server_key_version: "encryptionkey@0"
I20260806 09:28:56.646006 27190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786008495763054-20932-0/minicluster-data/ts-1/wal/instance:
uuid: "2b574edc4ab14a538db7020e110976cc"
format_stamp: "Formatted at 2026-08-06 09:28:56 on dist-test-slave-45dx"
server_key: "007ea35b25176759e3e0e4cbfa65a9ac"
server_key_iv: "fbc961d80f2d118e31eec3ca37c3d4a7"
server_key_version: "encryptionkey@0"
I20260806 09:28:56.650974 27190 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 09:28:56.654475 27206 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:28:56.655678 27190 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260806 09:28:56.655884 27190 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786008495763054-20932-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786008495763054-20932-0/minicluster-data/ts-1/wal
uuid: "2b574edc4ab14a538db7020e110976cc"
format_stamp: "Formatted at 2026-08-06 09:28:56 on dist-test-slave-45dx"
server_key: "007ea35b25176759e3e0e4cbfa65a9ac"
server_key_iv: "fbc961d80f2d118e31eec3ca37c3d4a7"
server_key_version: "encryptionkey@0"
I20260806 09:28:56.656100 27190 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786008495763054-20932-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786008495763054-20932-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786008495763054-20932-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:28:56.670737 27190 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:28:56.674970 27190 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:28:56.675266 27190 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:28:56.676155 27190 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:28:56.677481 27190 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:28:56.677603 27190 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:28:56.677735 27190 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:28:56.677830 27190 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:28:56.711953 27190 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.2:44033
I20260806 09:28:56.712014 27368 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.2:44033 every 8 connection(s)
I20260806 09:28:56.712882 27190 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786008495763054-20932-0/minicluster-data/ts-1/data/info.pb
I20260806 09:28:56.716216 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 27190
I20260806 09:28:56.716452 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786008495763054-20932-0/minicluster-data/ts-1/wal/instance
I20260806 09:28:56.716933 20932 external_mini_cluster.cc:1428] Setting key 2a5489710f3d4d73c9cacee1d04f8386
Aug 06 09:28:56 dist-test-slave-45dx krb5kdc[26847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008536, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.2@KRBTEST.COM for kudu/127.20.113.62@KRBTEST.COM
I20260806 09:28:56.735018 27371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:56.715409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.2:51711 (local address 127.20.113.62:42681)
0806 09:28:56.715856 (+ 447us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:56.715860 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:56.717096 (+ 1236us) server_negotiation.cc:407] Connection header received
0806 09:28:56.719245 (+ 2149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:56.719252 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:56.719338 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:56.719454 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:28:56.720912 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:56.722080 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:56.723572 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:56.724320 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:56.724480 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:56.728530 (+ 4050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:28:56.728538 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:28:56.728545 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:28:56.728589 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:28:56.731540 (+ 2951us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:56.732576 (+ 1036us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:56.732580 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:56.732583 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:56.732709 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:56.733235 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:56.733238 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:56.733240 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:56.733495 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:28:56.733606 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:56.734581 (+ 975us) server_negotiation.cc:299] Negotiation successful
0806 09:28:56.734741 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":250,"threads_started":1}
I20260806 09:28:56.736611 27373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:56.716019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:42681 (local address 127.20.113.2:51711)
0806 09:28:56.716859 (+ 840us) negotiation.cc:107] Waiting for socket to connect
0806 09:28:56.716927 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0806 09:28:56.718946 (+ 2019us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:28:56.719643 (+ 697us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:28:56.719651 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:28:56.720139 (+ 488us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:28:56.720739 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:56.720756 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:56.722223 (+ 1467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:56.722227 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:56.723431 (+ 1204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:56.723442 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:56.724567 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:56.724572 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:56.724746 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:56.725498 (+ 752us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:28:56.725535 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:28:56.728260 (+ 2725us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:28:56.731686 (+ 3426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:28:56.731697 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:28:56.731715 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:28:56.732457 (+ 742us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:28:56.732819 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:28:56.732827 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:28:56.732833 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:28:56.733109 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:28:56.733629 (+ 520us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:28:56.733639 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:28:56.734122 (+ 483us) client_negotiation.cc:769] Sending connection context
0806 09:28:56.734489 (+ 367us) client_negotiation.cc:240] Negotiation successful
0806 09:28:56.735079 (+ 590us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":578,"thread_start_us":389,"threads_started":1}
I20260806 09:28:56.738101 27369 heartbeater.cc:347] Connected to a master server at 127.20.113.62:42681
I20260806 09:28:56.738415 27369 heartbeater.cc:464] Registering TS with master...
I20260806 09:28:56.739090 27369 heartbeater.cc:511] Master 127.20.113.62:42681 requested a full tablet report, sending...
I20260806 09:28:56.740592 26909 ts_manager.cc:194] Registered new tserver with Master: 2b574edc4ab14a538db7020e110976cc (127.20.113.2:44033)
I20260806 09:28:56.741498 26909 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.113.2@KRBTEST.COM'} at 127.20.113.2:51711
WARNING: no policy specified for kudu/127.20.113.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.113.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.113.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.3.keytab.
Entry for principal kudu/127.20.113.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.3.keytab.
I20260806 09:28:56.775484 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786008495763054-20932-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786008495763054-20932-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786008495763054-20932-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786008495763054-20932-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.113.3:0
--local_ip_for_outbound_sockets=127.20.113.3
--webserver_interface=127.20.113.3
--webserver_port=0
--tserver_master_addrs=127.20.113.62:42681
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.3.keytab
--principal=kudu/127.20.113.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:36669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:28:57.115634 27376 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:28:57.115934 27376 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:28:57.116117 27376 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:28:57.123073 27376 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:28:57.123220 27376 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:28:57.123396 27376 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.113.3
I20260806 09:28:57.133400 27376 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:36669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786008495763054-20932-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786008495763054-20932-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.3.keytab
--principal=kudu/127.20.113.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.113.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786008495763054-20932-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.113.3
--webserver_port=0
--tserver_master_addrs=127.20.113.62:42681
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.113.3
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786008495763054-20932-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:28:57.134613 27376 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:28:57.135926 27376 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:28:57.148386 27386 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:28:57.148437 27390 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:28:57.148461 27384 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:28:57.149622 27376 server_base.cc:1034] running on GCE node
I20260806 09:28:57.150189 27376 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:28:57.151110 27376 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:28:57.152422 27376 hybrid_clock.cc:648] HybridClock initialized: now 1786008537152354 us; error 48 us; skew 500 ppm
I20260806 09:28:57.152738 27376 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:28:57 dist-test-slave-45dx krb5kdc[26847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008537, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:28:57.155879 27376 init.cc:377] Logged in from keytab as kudu/127.20.113.3@KRBTEST.COM (short username kudu)
I20260806 09:28:57.156973 27376 fs_manager.cc:359] Metadata directory not provided
I20260806 09:28:57.157105 27376 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:28:57.157425 27376 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:28:57.160307 27376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786008495763054-20932-0/minicluster-data/ts-2/data/instance:
uuid: "8f88d082fae545dfa0fc69ff733a1a71"
format_stamp: "Formatted at 2026-08-06 09:28:57 on dist-test-slave-45dx"
server_key: "7446caf9b593a10d7e3fb59ba986a268"
server_key_iv: "f8a5f759c7a4830d578b36495a9d2b34"
server_key_version: "encryptionkey@0"
I20260806 09:28:57.161151 27376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786008495763054-20932-0/minicluster-data/ts-2/wal/instance:
uuid: "8f88d082fae545dfa0fc69ff733a1a71"
format_stamp: "Formatted at 2026-08-06 09:28:57 on dist-test-slave-45dx"
server_key: "7446caf9b593a10d7e3fb59ba986a268"
server_key_iv: "f8a5f759c7a4830d578b36495a9d2b34"
server_key_version: "encryptionkey@0"
I20260806 09:28:57.166062 27376 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 09:28:57.169312 27397 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:28:57.170536 27376 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 09:28:57.170807 27376 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786008495763054-20932-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786008495763054-20932-0/minicluster-data/ts-2/wal
uuid: "8f88d082fae545dfa0fc69ff733a1a71"
format_stamp: "Formatted at 2026-08-06 09:28:57 on dist-test-slave-45dx"
server_key: "7446caf9b593a10d7e3fb59ba986a268"
server_key_iv: "f8a5f759c7a4830d578b36495a9d2b34"
server_key_version: "encryptionkey@0"
I20260806 09:28:57.171043 27376 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786008495763054-20932-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786008495763054-20932-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786008495763054-20932-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:28:57.188663 27376 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:28:57.193310 27376 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:28:57.193619 27376 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:28:57.194599 27376 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:28:57.195982 27376 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:28:57.196128 27376 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:28:57.196282 27376 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:28:57.196424 27376 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260806 09:28:57.224100 27181 heartbeater.cc:503] Master 127.20.113.62:42681 was elected leader, sending a full tablet report...
I20260806 09:28:57.231950 27376 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.3:44011
I20260806 09:28:57.232051 27559 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.3:44011 every 8 connection(s)
I20260806 09:28:57.232936 27376 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786008495763054-20932-0/minicluster-data/ts-2/data/info.pb
I20260806 09:28:57.238489 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 27376
I20260806 09:28:57.238686 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786008495763054-20932-0/minicluster-data/ts-2/wal/instance
I20260806 09:28:57.239205 20932 external_mini_cluster.cc:1428] Setting key 5e6ce0d39fb98b2754159fb183ac8842
Aug 06 09:28:57 dist-test-slave-45dx krb5kdc[26847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008537, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.3@KRBTEST.COM for kudu/127.20.113.62@KRBTEST.COM
I20260806 09:28:57.253453 27562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:57.235547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.3:54405 (local address 127.20.113.62:42681)
0806 09:28:57.236014 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:57.236019 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:57.237159 (+ 1140us) server_negotiation.cc:407] Connection header received
0806 09:28:57.239434 (+ 2275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:57.239439 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:57.239520 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:57.239620 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:28:57.241350 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:57.242396 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:57.243927 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:57.244444 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:57.244541 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:57.247782 (+ 3241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:28:57.247789 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:28:57.247793 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:28:57.247826 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:28:57.250509 (+ 2683us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:57.251159 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:57.251162 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:57.251165 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:57.251302 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:57.251682 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:57.251685 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:57.251687 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:57.251968 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:28:57.252093 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:57.252843 (+ 750us) server_negotiation.cc:299] Negotiation successful
0806 09:28:57.253100 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354,"thread_start_us":250,"threads_started":1}
I20260806 09:28:57.254462 27563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:57.236217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:42681 (local address 127.20.113.3:54405)
0806 09:28:57.237010 (+ 793us) negotiation.cc:107] Waiting for socket to connect
0806 09:28:57.237046 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 09:28:57.239113 (+ 2067us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:28:57.239876 (+ 763us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:28:57.239885 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:28:57.240446 (+ 561us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:28:57.241162 (+ 716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:57.241188 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:57.242553 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:57.242557 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:57.243792 (+ 1235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:57.243804 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:57.244640 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:57.244645 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:57.244812 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:57.245495 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:28:57.245515 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:28:57.247577 (+ 2062us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:28:57.250640 (+ 3063us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:28:57.250646 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:28:57.250656 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:28:57.251057 (+ 401us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:28:57.251403 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:28:57.251406 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:28:57.251408 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:28:57.251594 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:28:57.252087 (+ 493us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:28:57.252092 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:28:57.252453 (+ 361us) client_negotiation.cc:769] Sending connection context
0806 09:28:57.252739 (+ 286us) client_negotiation.cc:240] Negotiation successful
0806 09:28:57.253234 (+ 495us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":553,"thread_start_us":292,"threads_started":1}
I20260806 09:28:57.255651 27560 heartbeater.cc:347] Connected to a master server at 127.20.113.62:42681
I20260806 09:28:57.255970 27560 heartbeater.cc:464] Registering TS with master...
I20260806 09:28:57.256670 27560 heartbeater.cc:511] Master 127.20.113.62:42681 requested a full tablet report, sending...
I20260806 09:28:57.258005 26909 ts_manager.cc:194] Registered new tserver with Master: 8f88d082fae545dfa0fc69ff733a1a71 (127.20.113.3:44011)
I20260806 09:28:57.258716 26909 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.113.3@KRBTEST.COM'} at 127.20.113.3:54405
I20260806 09:28:57.266686 20932 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 09:28:57.283066 27562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:57.271186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57750 (local address 127.20.113.62:42681)
0806 09:28:57.271352 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:57.271357 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:57.271890 (+ 533us) server_negotiation.cc:407] Connection header received
0806 09:28:57.272004 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:57.272007 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:57.272066 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:57.272143 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:28:57.273552 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:57.274307 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:57.275609 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:57.276066 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:57.276161 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:57.278289 (+ 2128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:28:57.278294 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:28:57.278297 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:28:57.278327 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:28:57.280876 (+ 2549us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:57.281440 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:57.281444 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:57.281446 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:57.281558 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:57.281910 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:57.281914 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:57.281915 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:57.282213 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:28:57.282404 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:57.282665 (+ 261us) server_negotiation.cc:299] Negotiation successful
0806 09:28:57.282837 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 09:28:57.286947 26909 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57750:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 09:28:57.293567 26909 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 09:28:57.317191 27578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:57.305109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.2:44033 (local address 127.0.0.1:45936)
0806 09:28:57.306660 (+ 1551us) negotiation.cc:107] Waiting for socket to connect
0806 09:28:57.306701 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 09:28:57.306880 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:28:57.310466 (+ 3586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:28:57.310470 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:28:57.310495 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:28:57.310832 (+ 337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:57.310841 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:57.312505 (+ 1664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:57.312510 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:57.314872 (+ 2362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:57.314887 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:57.316151 (+ 1264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:57.316156 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:57.316354 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:57.316419 (+ 65us) client_negotiation.cc:769] Sending connection context
0806 09:28:57.316551 (+ 132us) client_negotiation.cc:240] Negotiation successful
0806 09:28:57.316723 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1343,"mutex_wait_us":73,"thread_start_us":1225,"threads_started":1}
I20260806 09:28:57.317549 27583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:57.307656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.3:44011 (local address 127.0.0.1:45632)
0806 09:28:57.308396 (+ 740us) negotiation.cc:107] Waiting for socket to connect
0806 09:28:57.308417 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 09:28:57.308604 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:28:57.309880 (+ 1276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:28:57.309885 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:28:57.309916 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:28:57.310378 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:57.310390 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:57.312549 (+ 2159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:57.312554 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:57.314872 (+ 2318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:57.314887 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:57.316610 (+ 1723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:57.316615 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:57.316753 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:57.316778 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 09:28:57.316868 (+ 90us) client_negotiation.cc:240] Negotiation successful
0806 09:28:57.317017 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":592,"mutex_wait_us":126,"spinlock_wait_cycles":101120,"thread_start_us":788,"threads_started":1}
I20260806 09:28:57.318032 27581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:57.305493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.1:42057 (local address 127.0.0.1:52618)
0806 09:28:57.308026 (+ 2533us) negotiation.cc:107] Waiting for socket to connect
0806 09:28:57.308051 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 09:28:57.308585 (+ 534us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:28:57.309479 (+ 894us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:28:57.309489 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:28:57.309515 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:28:57.309913 (+ 398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:57.309926 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:57.311874 (+ 1948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:57.311879 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:57.314481 (+ 2602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:57.314493 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:57.317154 (+ 2661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:57.317163 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:57.317380 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:57.317411 (+ 31us) client_negotiation.cc:769] Sending connection context
0806 09:28:57.317495 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 09:28:57.317618 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2394,"mutex_wait_us":1,"thread_start_us":2202,"threads_started":1}
I20260806 09:28:57.318218 27580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:57.305368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45936 (local address 127.20.113.2:44033)
0806 09:28:57.309994 (+ 4626us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:57.310005 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:57.310032 (+ 27us) server_negotiation.cc:407] Connection header received
0806 09:28:57.310134 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:57.310140 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:57.310291 (+ 151us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:57.310476 (+ 185us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:28:57.311007 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:57.312326 (+ 1319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:57.315077 (+ 2751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:57.315945 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:57.316099 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:57.316303 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:57.317713 (+ 1410us) server_negotiation.cc:299] Negotiation successful
0806 09:28:57.317885 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4470,"thread_start_us":4308,"threads_started":1}
I20260806 09:28:57.318379 27584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:57.308264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45632 (local address 127.20.113.3:44011)
0806 09:28:57.309240 (+ 976us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:57.309250 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:57.309285 (+ 35us) server_negotiation.cc:407] Connection header received
0806 09:28:57.309387 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:57.309397 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:57.309630 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:57.310571 (+ 941us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:28:57.310855 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:57.312390 (+ 1535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:57.315042 (+ 2652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:57.316408 (+ 1366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:57.316533 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:57.316735 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:57.317240 (+ 505us) server_negotiation.cc:299] Negotiation successful
0806 09:28:57.317807 (+ 567us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":773,"thread_start_us":1034,"threads_started":1}
I20260806 09:28:57.318818 27582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:57.306284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52618 (local address 127.20.113.1:42057)
0806 09:28:57.307540 (+ 1256us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:57.307550 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:57.308501 (+ 951us) server_negotiation.cc:407] Connection header received
0806 09:28:57.308792 (+ 291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:57.308799 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:57.309033 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:57.309218 (+ 185us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:28:57.310112 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:57.311689 (+ 1577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:57.314636 (+ 2947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:57.315986 (+ 1350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:57.316880 (+ 894us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:57.317129 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:57.318311 (+ 1182us) server_negotiation.cc:299] Negotiation successful
0806 09:28:57.318499 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1058,"thread_start_us":917,"threads_started":1}
I20260806 09:28:57.321969 27465 tablet_service.cc:1467] Processing CreateTablet for tablet 00ccd9c35a9e47d2aaf147290d1447ec (DEFAULT_TABLE table=test-table [id=91358b009425425a82610ac1a3b09cbc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:28:57.321995 27270 tablet_service.cc:1467] Processing CreateTablet for tablet 00ccd9c35a9e47d2aaf147290d1447ec (DEFAULT_TABLE table=test-table [id=91358b009425425a82610ac1a3b09cbc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:28:57.322263 27092 tablet_service.cc:1467] Processing CreateTablet for tablet 00ccd9c35a9e47d2aaf147290d1447ec (DEFAULT_TABLE table=test-table [id=91358b009425425a82610ac1a3b09cbc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:28:57.323292 27465 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00ccd9c35a9e47d2aaf147290d1447ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:28:57.323441 27270 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00ccd9c35a9e47d2aaf147290d1447ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:28:57.323513 27092 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00ccd9c35a9e47d2aaf147290d1447ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:28:57.336217 27585 tablet_bootstrap.cc:492] T 00ccd9c35a9e47d2aaf147290d1447ec P 8f88d082fae545dfa0fc69ff733a1a71: Bootstrap starting.
I20260806 09:28:57.338419 27588 tablet_bootstrap.cc:492] T 00ccd9c35a9e47d2aaf147290d1447ec P 2b574edc4ab14a538db7020e110976cc: Bootstrap starting.
I20260806 09:28:57.340420 27585 tablet_bootstrap.cc:654] T 00ccd9c35a9e47d2aaf147290d1447ec P 8f88d082fae545dfa0fc69ff733a1a71: Neither blocks nor log segments found. Creating new log.
I20260806 09:28:57.341053 27587 tablet_bootstrap.cc:492] T 00ccd9c35a9e47d2aaf147290d1447ec P e2e792c28ea3454cbaa6dda6d86593f3: Bootstrap starting.
I20260806 09:28:57.342162 27588 tablet_bootstrap.cc:654] T 00ccd9c35a9e47d2aaf147290d1447ec P 2b574edc4ab14a538db7020e110976cc: Neither blocks nor log segments found. Creating new log.
I20260806 09:28:57.343425 27588 log.cc:824] T 00ccd9c35a9e47d2aaf147290d1447ec P 2b574edc4ab14a538db7020e110976cc: Log is configured to *not* fsync() on all Append() calls
I20260806 09:28:57.343783 27585 log.cc:824] T 00ccd9c35a9e47d2aaf147290d1447ec P 8f88d082fae545dfa0fc69ff733a1a71: Log is configured to *not* fsync() on all Append() calls
I20260806 09:28:57.344610 27587 tablet_bootstrap.cc:654] T 00ccd9c35a9e47d2aaf147290d1447ec P e2e792c28ea3454cbaa6dda6d86593f3: Neither blocks nor log segments found. Creating new log.
I20260806 09:28:57.345835 27587 log.cc:824] T 00ccd9c35a9e47d2aaf147290d1447ec P e2e792c28ea3454cbaa6dda6d86593f3: Log is configured to *not* fsync() on all Append() calls
I20260806 09:28:57.346783 27588 tablet_bootstrap.cc:492] T 00ccd9c35a9e47d2aaf147290d1447ec P 2b574edc4ab14a538db7020e110976cc: No bootstrap required, opened a new log
I20260806 09:28:57.347123 27585 tablet_bootstrap.cc:492] T 00ccd9c35a9e47d2aaf147290d1447ec P 8f88d082fae545dfa0fc69ff733a1a71: No bootstrap required, opened a new log
I20260806 09:28:57.347103 27588 ts_tablet_manager.cc:1397] T 00ccd9c35a9e47d2aaf147290d1447ec P 2b574edc4ab14a538db7020e110976cc: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.006s
I20260806 09:28:57.347388 27585 ts_tablet_manager.cc:1397] T 00ccd9c35a9e47d2aaf147290d1447ec P 8f88d082fae545dfa0fc69ff733a1a71: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.004s
I20260806 09:28:57.348882 27587 tablet_bootstrap.cc:492] T 00ccd9c35a9e47d2aaf147290d1447ec P e2e792c28ea3454cbaa6dda6d86593f3: No bootstrap required, opened a new log
I20260806 09:28:57.349169 27587 ts_tablet_manager.cc:1397] T 00ccd9c35a9e47d2aaf147290d1447ec P e2e792c28ea3454cbaa6dda6d86593f3: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260806 09:28:57.355088 27585 raft_consensus.cc:348] T 00ccd9c35a9e47d2aaf147290d1447ec P 8f88d082fae545dfa0fc69ff733a1a71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b574edc4ab14a538db7020e110976cc" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 44033 } } peers { permanent_uuid: "e2e792c28ea3454cbaa6dda6d86593f3" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 42057 } } peers { permanent_uuid: "8f88d082fae545dfa0fc69ff733a1a71" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 44011 } }
I20260806 09:28:57.355454 27585 raft_consensus.cc:374] T 00ccd9c35a9e47d2aaf147290d1447ec P 8f88d082fae545dfa0fc69ff733a1a71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:28:57.355583 27585 raft_consensus.cc:729] T 00ccd9c35a9e47d2aaf147290d1447ec P 8f88d082fae545dfa0fc69ff733a1a71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f88d082fae545dfa0fc69ff733a1a71, State: Initialized, Role: FOLLOWER
I20260806 09:28:57.356199 27585 consensus_queue.cc:260] T 00ccd9c35a9e47d2aaf147290d1447ec P 8f88d082fae545dfa0fc69ff733a1a71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b574edc4ab14a538db7020e110976cc" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 44033 } } peers { permanent_uuid: "e2e792c28ea3454cbaa6dda6d86593f3" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 42057 } } peers { permanent_uuid: "8f88d082fae545dfa0fc69ff733a1a71" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 44011 } }
I20260806 09:28:57.357412 27560 heartbeater.cc:503] Master 127.20.113.62:42681 was elected leader, sending a full tablet report...
I20260806 09:28:57.357882 27585 ts_tablet_manager.cc:1428] T 00ccd9c35a9e47d2aaf147290d1447ec P 8f88d082fae545dfa0fc69ff733a1a71: Time spent starting tablet: real 0.010s user 0.006s sys 0.005s
I20260806 09:28:57.357692 27588 raft_consensus.cc:348] T 00ccd9c35a9e47d2aaf147290d1447ec P 2b574edc4ab14a538db7020e110976cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b574edc4ab14a538db7020e110976cc" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 44033 } } peers { permanent_uuid: "e2e792c28ea3454cbaa6dda6d86593f3" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 42057 } } peers { permanent_uuid: "8f88d082fae545dfa0fc69ff733a1a71" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 44011 } }
I20260806 09:28:57.358114 27588 raft_consensus.cc:374] T 00ccd9c35a9e47d2aaf147290d1447ec P 2b574edc4ab14a538db7020e110976cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:28:57.358227 27588 raft_consensus.cc:729] T 00ccd9c35a9e47d2aaf147290d1447ec P 2b574edc4ab14a538db7020e110976cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b574edc4ab14a538db7020e110976cc, State: Initialized, Role: FOLLOWER
I20260806 09:28:57.358807 27588 consensus_queue.cc:260] T 00ccd9c35a9e47d2aaf147290d1447ec P 2b574edc4ab14a538db7020e110976cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b574edc4ab14a538db7020e110976cc" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 44033 } } peers { permanent_uuid: "e2e792c28ea3454cbaa6dda6d86593f3" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 42057 } } peers { permanent_uuid: "8f88d082fae545dfa0fc69ff733a1a71" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 44011 } }
I20260806 09:28:57.359927 27369 heartbeater.cc:503] Master 127.20.113.62:42681 was elected leader, sending a full tablet report...
I20260806 09:28:57.361869 27588 ts_tablet_manager.cc:1428] T 00ccd9c35a9e47d2aaf147290d1447ec P 2b574edc4ab14a538db7020e110976cc: Time spent starting tablet: real 0.015s user 0.015s sys 0.000s
I20260806 09:28:57.361334 27587 raft_consensus.cc:348] T 00ccd9c35a9e47d2aaf147290d1447ec P e2e792c28ea3454cbaa6dda6d86593f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b574edc4ab14a538db7020e110976cc" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 44033 } } peers { permanent_uuid: "e2e792c28ea3454cbaa6dda6d86593f3" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 42057 } } peers { permanent_uuid: "8f88d082fae545dfa0fc69ff733a1a71" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 44011 } }
I20260806 09:28:57.362315 27587 raft_consensus.cc:374] T 00ccd9c35a9e47d2aaf147290d1447ec P e2e792c28ea3454cbaa6dda6d86593f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:28:57.362452 27587 raft_consensus.cc:729] T 00ccd9c35a9e47d2aaf147290d1447ec P e2e792c28ea3454cbaa6dda6d86593f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2e792c28ea3454cbaa6dda6d86593f3, State: Initialized, Role: FOLLOWER
I20260806 09:28:57.363101 27587 consensus_queue.cc:260] T 00ccd9c35a9e47d2aaf147290d1447ec P e2e792c28ea3454cbaa6dda6d86593f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b574edc4ab14a538db7020e110976cc" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 44033 } } peers { permanent_uuid: "e2e792c28ea3454cbaa6dda6d86593f3" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 42057 } } peers { permanent_uuid: "8f88d082fae545dfa0fc69ff733a1a71" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 44011 } }
I20260806 09:28:57.365454 27587 ts_tablet_manager.cc:1428] T 00ccd9c35a9e47d2aaf147290d1447ec P e2e792c28ea3454cbaa6dda6d86593f3: Time spent starting tablet: real 0.016s user 0.012s sys 0.002s
W20260806 09:28:57.442966 27183 tablet.cc:2369] T 00ccd9c35a9e47d2aaf147290d1447ec P e2e792c28ea3454cbaa6dda6d86593f3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 09:28:57.452617 27596 raft_consensus.cc:482] T 00ccd9c35a9e47d2aaf147290d1447ec P 2b574edc4ab14a538db7020e110976cc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 09:28:57.452934 27596 raft_consensus.cc:504] T 00ccd9c35a9e47d2aaf147290d1447ec P 2b574edc4ab14a538db7020e110976cc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b574edc4ab14a538db7020e110976cc" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 44033 } } peers { permanent_uuid: "e2e792c28ea3454cbaa6dda6d86593f3" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 42057 } } peers { permanent_uuid: "8f88d082fae545dfa0fc69ff733a1a71" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 44011 } }
I20260806 09:28:57.454656 27596 leader_election.cc:288] T 00ccd9c35a9e47d2aaf147290d1447ec P 2b574edc4ab14a538db7020e110976cc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e2e792c28ea3454cbaa6dda6d86593f3 (127.20.113.1:42057), 8f88d082fae545dfa0fc69ff733a1a71 (127.20.113.3:44011)
I20260806 09:28:57.463799 27582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:57.454968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.2:48727 (local address 127.20.113.1:42057)
0806 09:28:57.455175 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:57.455185 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:57.455932 (+ 747us) server_negotiation.cc:407] Connection header received
0806 09:28:57.456170 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:57.456174 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:57.456234 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:57.456350 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:28:57.457093 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:57.458255 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:57.461380 (+ 3125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:57.462623 (+ 1243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:57.462815 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:57.462894 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:57.463383 (+ 489us) server_negotiation.cc:299] Negotiation successful
0806 09:28:57.463546 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 09:28:57.463888 27584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:57.455226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.2:46045 (local address 127.20.113.3:44011)
0806 09:28:57.455470 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:57.455479 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:57.456198 (+ 719us) server_negotiation.cc:407] Connection header received
0806 09:28:57.456432 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:57.456436 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:57.456521 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:57.456615 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:28:57.457339 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:57.458411 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:57.461370 (+ 2959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:57.462648 (+ 1278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:57.462835 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:57.462900 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:57.463512 (+ 612us) server_negotiation.cc:299] Negotiation successful
0806 09:28:57.463640 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 09:28:57.463984 27604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:57.454888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.1:42057 (local address 127.20.113.2:48727)
0806 09:28:57.455809 (+ 921us) negotiation.cc:107] Waiting for socket to connect
0806 09:28:57.455827 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 09:28:57.456011 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:28:57.456381 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:28:57.456385 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:28:57.456411 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:28:57.456829 (+ 418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:57.456842 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:57.458421 (+ 1579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:57.458426 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:57.461193 (+ 2767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:57.461203 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:57.462900 (+ 1697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:57.462909 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:57.463161 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:57.463210 (+ 49us) client_negotiation.cc:769] Sending connection context
0806 09:28:57.463348 (+ 138us) client_negotiation.cc:240] Negotiation successful
0806 09:28:57.463525 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":791,"mutex_wait_us":208,"thread_start_us":528,"threads_started":1}
I20260806 09:28:57.464759 27606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:57.455047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.3:44011 (local address 127.20.113.2:46045)
0806 09:28:57.456089 (+ 1042us) negotiation.cc:107] Waiting for socket to connect
0806 09:28:57.456103 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 09:28:57.456247 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:28:57.456647 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:28:57.456654 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:28:57.456688 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:28:57.457172 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:57.457181 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:57.458564 (+ 1383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:57.458571 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:57.461186 (+ 2615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:28:57.461203 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:28:57.463088 (+ 1885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:28:57.463097 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:28:57.463321 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:57.463356 (+ 35us) client_negotiation.cc:769] Sending connection context
0806 09:28:57.464172 (+ 816us) client_negotiation.cc:240] Negotiation successful
0806 09:28:57.464337 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":885,"mutex_wait_us":70,"thread_start_us":635,"threads_started":1}
W20260806 09:28:57.464846 27370 tablet.cc:2369] T 00ccd9c35a9e47d2aaf147290d1447ec P 2b574edc4ab14a538db7020e110976cc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 09:28:57.465101 27117 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00ccd9c35a9e47d2aaf147290d1447ec" candidate_uuid: "2b574edc4ab14a538db7020e110976cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2e792c28ea3454cbaa6dda6d86593f3" is_pre_election: true
I20260806 09:28:57.465704 27117 raft_consensus.cc:2454] T 00ccd9c35a9e47d2aaf147290d1447ec P e2e792c28ea3454cbaa6dda6d86593f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b574edc4ab14a538db7020e110976cc in term 0.
I20260806 09:28:57.465929 27496 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00ccd9c35a9e47d2aaf147290d1447ec" candidate_uuid: "2b574edc4ab14a538db7020e110976cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f88d082fae545dfa0fc69ff733a1a71" is_pre_election: true
I20260806 09:28:57.466415 27211 leader_election.cc:302] T 00ccd9c35a9e47d2aaf147290d1447ec P 2b574edc4ab14a538db7020e110976cc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b574edc4ab14a538db7020e110976cc, e2e792c28ea3454cbaa6dda6d86593f3; no voters:
I20260806 09:28:57.466461 27496 raft_consensus.cc:2454] T 00ccd9c35a9e47d2aaf147290d1447ec P 8f88d082fae545dfa0fc69ff733a1a71 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b574edc4ab14a538db7020e110976cc in term 0.
I20260806 09:28:57.466832 27596 raft_consensus.cc:2781] T 00ccd9c35a9e47d2aaf147290d1447ec P 2b574edc4ab14a538db7020e110976cc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 09:28:57.467006 27596 raft_consensus.cc:482] T 00ccd9c35a9e47d2aaf147290d1447ec P 2b574edc4ab14a538db7020e110976cc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 09:28:57.467164 27596 raft_consensus.cc:3037] T 00ccd9c35a9e47d2aaf147290d1447ec P 2b574edc4ab14a538db7020e110976cc [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:28:57.469759 27596 raft_consensus.cc:504] T 00ccd9c35a9e47d2aaf147290d1447ec P 2b574edc4ab14a538db7020e110976cc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b574edc4ab14a538db7020e110976cc" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 44033 } } peers { permanent_uuid: "e2e792c28ea3454cbaa6dda6d86593f3" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 42057 } } peers { permanent_uuid: "8f88d082fae545dfa0fc69ff733a1a71" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 44011 } }
I20260806 09:28:57.470470 27596 leader_election.cc:288] T 00ccd9c35a9e47d2aaf147290d1447ec P 2b574edc4ab14a538db7020e110976cc [CANDIDATE]: Term 1 election: Requested vote from peers e2e792c28ea3454cbaa6dda6d86593f3 (127.20.113.1:42057), 8f88d082fae545dfa0fc69ff733a1a71 (127.20.113.3:44011)
I20260806 09:28:57.470852 27117 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00ccd9c35a9e47d2aaf147290d1447ec" candidate_uuid: "2b574edc4ab14a538db7020e110976cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2e792c28ea3454cbaa6dda6d86593f3"
I20260806 09:28:57.470980 27496 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00ccd9c35a9e47d2aaf147290d1447ec" candidate_uuid: "2b574edc4ab14a538db7020e110976cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f88d082fae545dfa0fc69ff733a1a71"
I20260806 09:28:57.471074 27117 raft_consensus.cc:3037] T 00ccd9c35a9e47d2aaf147290d1447ec P e2e792c28ea3454cbaa6dda6d86593f3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:28:57.471184 27496 raft_consensus.cc:3037] T 00ccd9c35a9e47d2aaf147290d1447ec P 8f88d082fae545dfa0fc69ff733a1a71 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:28:57.474443 27117 raft_consensus.cc:2454] T 00ccd9c35a9e47d2aaf147290d1447ec P e2e792c28ea3454cbaa6dda6d86593f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b574edc4ab14a538db7020e110976cc in term 1.
I20260806 09:28:57.474499 27496 raft_consensus.cc:2454] T 00ccd9c35a9e47d2aaf147290d1447ec P 8f88d082fae545dfa0fc69ff733a1a71 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b574edc4ab14a538db7020e110976cc in term 1.
I20260806 09:28:57.474893 27211 leader_election.cc:302] T 00ccd9c35a9e47d2aaf147290d1447ec P 2b574edc4ab14a538db7020e110976cc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b574edc4ab14a538db7020e110976cc, e2e792c28ea3454cbaa6dda6d86593f3; no voters:
I20260806 09:28:57.475256 27596 raft_consensus.cc:2781] T 00ccd9c35a9e47d2aaf147290d1447ec P 2b574edc4ab14a538db7020e110976cc [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:28:57.475724 27596 raft_consensus.cc:686] T 00ccd9c35a9e47d2aaf147290d1447ec P 2b574edc4ab14a538db7020e110976cc [term 1 LEADER]: Becoming Leader. State: Replica: 2b574edc4ab14a538db7020e110976cc, State: Running, Role: LEADER
I20260806 09:28:57.476419 27596 consensus_queue.cc:237] T 00ccd9c35a9e47d2aaf147290d1447ec P 2b574edc4ab14a538db7020e110976cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b574edc4ab14a538db7020e110976cc" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 44033 } } peers { permanent_uuid: "e2e792c28ea3454cbaa6dda6d86593f3" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 42057 } } peers { permanent_uuid: "8f88d082fae545dfa0fc69ff733a1a71" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 44011 } }
I20260806 09:28:57.481259 26909 catalog_manager.cc:5495] T 00ccd9c35a9e47d2aaf147290d1447ec P 2b574edc4ab14a538db7020e110976cc reported cstate change: term changed from 0 to 1, leader changed from <none> to 2b574edc4ab14a538db7020e110976cc (127.20.113.2). New cstate: current_term: 1 leader_uuid: "2b574edc4ab14a538db7020e110976cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b574edc4ab14a538db7020e110976cc" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 44033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e2e792c28ea3454cbaa6dda6d86593f3" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 42057 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8f88d082fae545dfa0fc69ff733a1a71" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 44011 } health_report { overall_health: UNKNOWN } } }
W20260806 09:28:57.484576 27561 tablet.cc:2369] T 00ccd9c35a9e47d2aaf147290d1447ec P 8f88d082fae545dfa0fc69ff733a1a71: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 09:28:57.507094 27580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:57.501443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45948 (local address 127.20.113.2:44033)
0806 09:28:57.501689 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:57.501698 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:57.501820 (+ 122us) server_negotiation.cc:407] Connection header received
0806 09:28:57.502083 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:57.502087 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:57.502173 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:57.502334 (+ 161us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 09:28:57.502761 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:57.503483 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:57.504930 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:57.505344 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:57.505484 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:57.505978 (+ 494us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 09:28:57.506473 (+ 495us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 09:28:57.506666 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:57.506743 (+ 77us) server_negotiation.cc:299] Negotiation successful
0806 09:28:57.506827 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260806 09:28:57.515312 27496 raft_consensus.cc:1261] T 00ccd9c35a9e47d2aaf147290d1447ec P 8f88d082fae545dfa0fc69ff733a1a71 [term 1 FOLLOWER]: Refusing update from remote peer 2b574edc4ab14a538db7020e110976cc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 09:28:57.515309 27117 raft_consensus.cc:1261] T 00ccd9c35a9e47d2aaf147290d1447ec P e2e792c28ea3454cbaa6dda6d86593f3 [term 1 FOLLOWER]: Refusing update from remote peer 2b574edc4ab14a538db7020e110976cc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 09:28:57.516273 27610 consensus_queue.cc:1035] T 00ccd9c35a9e47d2aaf147290d1447ec P 2b574edc4ab14a538db7020e110976cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f88d082fae545dfa0fc69ff733a1a71" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 44011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 09:28:57.516713 27596 consensus_queue.cc:1035] T 00ccd9c35a9e47d2aaf147290d1447ec P 2b574edc4ab14a538db7020e110976cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2e792c28ea3454cbaa6dda6d86593f3" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 42057 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 09:28:57.529346 27614 mvcc.cc:205] Tried to move back new op lower bound from 7315490969653129216 to 7315490969507102720. Current Snapshot: MvccSnapshot[applied={T|T < 7315490969653129216}]
I20260806 09:28:57.542582 26909 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57750:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 09:28:57.542879 26909 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57750:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 09:28:57.546196 26909 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7eb19a968c444ac8983e47c29260c23f: Sending DeleteTablet for 3 replicas of tablet 00ccd9c35a9e47d2aaf147290d1447ec
I20260806 09:28:57.547333 27270 tablet_service.cc:1514] Processing DeleteTablet for tablet 00ccd9c35a9e47d2aaf147290d1447ec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:28:57 UTC) from {username='kudu', principal='kudu/127.20.113.62@KRBTEST.COM'} at 127.0.0.1:45936
I20260806 09:28:57.547549 27092 tablet_service.cc:1514] Processing DeleteTablet for tablet 00ccd9c35a9e47d2aaf147290d1447ec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:28:57 UTC) from {username='kudu', principal='kudu/127.20.113.62@KRBTEST.COM'} at 127.0.0.1:52618
I20260806 09:28:57.548286 27625 tablet_replica.cc:330] T 00ccd9c35a9e47d2aaf147290d1447ec P 2b574edc4ab14a538db7020e110976cc: stopping tablet replica
I20260806 09:28:57.548480 27626 tablet_replica.cc:330] T 00ccd9c35a9e47d2aaf147290d1447ec P e2e792c28ea3454cbaa6dda6d86593f3: stopping tablet replica
I20260806 09:28:57.548516 27465 tablet_service.cc:1514] Processing DeleteTablet for tablet 00ccd9c35a9e47d2aaf147290d1447ec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:28:57 UTC) from {username='kudu', principal='kudu/127.20.113.62@KRBTEST.COM'} at 127.0.0.1:45632
I20260806 09:28:57.548895 27626 raft_consensus.cc:2229] T 00ccd9c35a9e47d2aaf147290d1447ec P e2e792c28ea3454cbaa6dda6d86593f3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 09:28:57.548910 27625 raft_consensus.cc:2229] T 00ccd9c35a9e47d2aaf147290d1447ec P 2b574edc4ab14a538db7020e110976cc [term 1 LEADER]: Raft consensus shutting down.
I20260806 09:28:57.549401 27626 raft_consensus.cc:2258] T 00ccd9c35a9e47d2aaf147290d1447ec P e2e792c28ea3454cbaa6dda6d86593f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 09:28:57.549657 27625 raft_consensus.cc:2258] T 00ccd9c35a9e47d2aaf147290d1447ec P 2b574edc4ab14a538db7020e110976cc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 09:28:57.550429 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 26999
I20260806 09:28:57.551467 27626 ts_tablet_manager.cc:1905] T 00ccd9c35a9e47d2aaf147290d1447ec P e2e792c28ea3454cbaa6dda6d86593f3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 09:28:57.552078 27625 ts_tablet_manager.cc:1905] T 00ccd9c35a9e47d2aaf147290d1447ec P 2b574edc4ab14a538db7020e110976cc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 09:28:57.559583 27625 ts_tablet_manager.cc:1918] T 00ccd9c35a9e47d2aaf147290d1447ec P 2b574edc4ab14a538db7020e110976cc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 09:28:57.559842 27625 log.cc:1196] T 00ccd9c35a9e47d2aaf147290d1447ec P 2b574edc4ab14a538db7020e110976cc: Deleting WAL directory at /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786008495763054-20932-0/minicluster-data/ts-1/wal/wals/00ccd9c35a9e47d2aaf147290d1447ec
I20260806 09:28:57.559947 27626 ts_tablet_manager.cc:1918] T 00ccd9c35a9e47d2aaf147290d1447ec P e2e792c28ea3454cbaa6dda6d86593f3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 09:28:57.560139 27626 log.cc:1196] T 00ccd9c35a9e47d2aaf147290d1447ec P e2e792c28ea3454cbaa6dda6d86593f3: Deleting WAL directory at /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786008495763054-20932-0/minicluster-data/ts-0/wal/wals/00ccd9c35a9e47d2aaf147290d1447ec
I20260806 09:28:57.560674 27625 ts_tablet_manager.cc:1939] T 00ccd9c35a9e47d2aaf147290d1447ec P 2b574edc4ab14a538db7020e110976cc: Deleting consensus metadata
I20260806 09:28:57.560860 27626 ts_tablet_manager.cc:1939] T 00ccd9c35a9e47d2aaf147290d1447ec P e2e792c28ea3454cbaa6dda6d86593f3: Deleting consensus metadata
I20260806 09:28:57.562541 26886 catalog_manager.cc:4841] TS 2b574edc4ab14a538db7020e110976cc (127.20.113.2:44033): tablet 00ccd9c35a9e47d2aaf147290d1447ec (table test-table [id=91358b009425425a82610ac1a3b09cbc]) successfully deleted
I20260806 09:28:57.563042 26889 catalog_manager.cc:4841] TS e2e792c28ea3454cbaa6dda6d86593f3 (127.20.113.1:42057): tablet 00ccd9c35a9e47d2aaf147290d1447ec (table test-table [id=91358b009425425a82610ac1a3b09cbc]) successfully deleted
I20260806 09:28:57.565610 27627 tablet_replica.cc:330] T 00ccd9c35a9e47d2aaf147290d1447ec P 8f88d082fae545dfa0fc69ff733a1a71: stopping tablet replica
I20260806 09:28:57.566092 27627 raft_consensus.cc:2229] T 00ccd9c35a9e47d2aaf147290d1447ec P 8f88d082fae545dfa0fc69ff733a1a71 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 09:28:57.566635 27627 raft_consensus.cc:2258] T 00ccd9c35a9e47d2aaf147290d1447ec P 8f88d082fae545dfa0fc69ff733a1a71 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 09:28:57.569571 27627 ts_tablet_manager.cc:1905] T 00ccd9c35a9e47d2aaf147290d1447ec P 8f88d082fae545dfa0fc69ff733a1a71: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 06 09:28:57 dist-test-slave-45dx krb5kdc[26847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008535, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.113.1@KRBTEST.COM
I20260806 09:28:57.575940 27627 ts_tablet_manager.cc:1918] T 00ccd9c35a9e47d2aaf147290d1447ec P 8f88d082fae545dfa0fc69ff733a1a71: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 09:28:57.576107 27627 log.cc:1196] T 00ccd9c35a9e47d2aaf147290d1447ec P 8f88d082fae545dfa0fc69ff733a1a71: Deleting WAL directory at /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786008495763054-20932-0/minicluster-data/ts-2/wal/wals/00ccd9c35a9e47d2aaf147290d1447ec
I20260806 09:28:57.576707 27627 ts_tablet_manager.cc:1939] T 00ccd9c35a9e47d2aaf147290d1447ec P 8f88d082fae545dfa0fc69ff733a1a71: Deleting consensus metadata
I20260806 09:28:57.578639 26885 catalog_manager.cc:4841] TS 8f88d082fae545dfa0fc69ff733a1a71 (127.20.113.3:44011): tablet 00ccd9c35a9e47d2aaf147290d1447ec (table test-table [id=91358b009425425a82610ac1a3b09cbc]) successfully deleted
I20260806 09:28:57.582302 27582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:57.553009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52622 (local address 127.20.113.1:42057)
0806 09:28:57.553179 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:57.553186 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:57.564855 (+ 11669us) server_negotiation.cc:407] Connection header received
0806 09:28:57.565090 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:57.565096 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:57.565181 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:57.565292 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:28:57.567068 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:57.567810 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:57.569610 (+ 1800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:57.570233 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:57.570366 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:57.575525 (+ 5159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:28:57.575540 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:28:57.575559 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:28:57.575602 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:28:57.579388 (+ 3786us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:57.580220 (+ 832us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:57.580227 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:57.580229 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:57.580434 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:57.580843 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:57.580846 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:57.580848 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:57.581210 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:28:57.581456 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:57.581807 (+ 351us) server_negotiation.cc:299] Negotiation successful
0806 09:28:57.581980 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 09:28:57.583019 27171 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:28:57.737919 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 26999
I20260806 09:28:57.762538 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 27190
Aug 06 09:28:57 dist-test-slave-45dx krb5kdc[26847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008535, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.113.2@KRBTEST.COM
I20260806 09:28:57.778726 27580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:57.763450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45958 (local address 127.20.113.2:44033)
0806 09:28:57.763641 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:57.763647 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:57.763668 (+ 21us) server_negotiation.cc:407] Connection header received
0806 09:28:57.763791 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:57.763796 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:57.763884 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:57.763993 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:28:57.765284 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:57.766179 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:57.767368 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:57.767812 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:57.767954 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:57.772252 (+ 4298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:28:57.772265 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:28:57.772279 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:28:57.772365 (+ 86us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:28:57.775757 (+ 3392us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:57.776581 (+ 824us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:57.776589 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:57.776592 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:57.776741 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:57.777145 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:57.777149 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:57.777151 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:57.777515 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:28:57.777886 (+ 371us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:57.778128 (+ 242us) server_negotiation.cc:299] Negotiation successful
0806 09:28:57.778370 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 09:28:57.779690 27362 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:28:57.895565 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 27190
I20260806 09:28:57.922072 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 27376
Aug 06 09:28:57 dist-test-slave-45dx krb5kdc[26847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008535, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.113.3@KRBTEST.COM
I20260806 09:28:57.937378 27584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:57.922883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45634 (local address 127.20.113.3:44011)
0806 09:28:57.923063 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:57.923067 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:57.923217 (+ 150us) server_negotiation.cc:407] Connection header received
0806 09:28:57.923393 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:57.923397 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:57.923463 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:57.923549 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:28:57.924917 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:57.925688 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:57.927132 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:28:57.927496 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:28:57.927590 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:28:57.932198 (+ 4608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:28:57.932227 (+ 29us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:28:57.932244 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:28:57.932280 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:28:57.934921 (+ 2641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:57.935511 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:57.935517 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:57.935519 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:57.935642 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:28:57.935987 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:28:57.935990 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:28:57.935991 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:28:57.936375 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:28:57.936626 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:57.936967 (+ 341us) server_negotiation.cc:299] Negotiation successful
0806 09:28:57.937125 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 09:28:57.938046 27550 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:28:58.027199 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 27376
I20260806 09:28:58.052205 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 26862
I20260806 09:28:58.053411 26949 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:28:58.164626 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 26862
2026-08-06T09:28:58Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4563 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-06T09:28:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T09:28:58Z Disabled control of system clock
I20260806 09:28:58.250897 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786008495763054-20932-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786008495763054-20932-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786008495763054-20932-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.113.62:38975
--webserver_interface=127.20.113.62
--webserver_port=0
--builtin_ntp_servers=127.20.113.20:34387
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.113.62:38975
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 09:28:58.583077 27639 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:28:58.583382 27639 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:28:58.583537 27639 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:28:58.591008 27639 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 09:28:58.591106 27639 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:28:58.591145 27639 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:28:58.591238 27639 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 09:28:58.591306 27639 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 09:28:58.600453 27639 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:34387
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786008495763054-20932-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786008495763054-20932-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.113.62:38975
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.113.62:38975
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.113.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786008495763054-20932-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:28:58.601792 27639 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:28:58.603648 27639 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:28:58.615516 27648 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:28:58.615553 27652 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:28:58.615609 27647 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:28:58.616434 27639 server_base.cc:1034] running on GCE node
I20260806 09:28:58.617288 27639 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:28:58.618650 27639 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:28:58.619901 27639 hybrid_clock.cc:648] HybridClock initialized: now 1786008538619856 us; error 59 us; skew 500 ppm
I20260806 09:28:58.620262 27639 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 09:28:58.622771 27639 webserver.cc:460] Webserver started at http://127.20.113.62:41757/ using document root <none> and password file <none>
I20260806 09:28:58.623523 27639 fs_manager.cc:359] Metadata directory not provided
I20260806 09:28:58.623656 27639 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:28:58.624017 27639 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:28:58.627014 27639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786008495763054-20932-0/minicluster-data/master-0/data/instance:
uuid: "e749c4a6f6294d4a800f80937d0413f9"
format_stamp: "Formatted at 2026-08-06 09:28:58 on dist-test-slave-45dx"
server_key: "64dd37dd1fff43a4e6705d624a7ffacd"
server_key_iv: "0a18cbb23ac7d76b20b8e42190bfc39b"
server_key_version: "encryptionkey@0"
I20260806 09:28:58.627818 27639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786008495763054-20932-0/minicluster-data/master-0/wal/instance:
uuid: "e749c4a6f6294d4a800f80937d0413f9"
format_stamp: "Formatted at 2026-08-06 09:28:58 on dist-test-slave-45dx"
server_key: "64dd37dd1fff43a4e6705d624a7ffacd"
server_key_iv: "0a18cbb23ac7d76b20b8e42190bfc39b"
server_key_version: "encryptionkey@0"
I20260806 09:28:58.633049 27639 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260806 09:28:58.636585 27660 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:28:58.637826 27639 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260806 09:28:58.638019 27639 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786008495763054-20932-0/minicluster-data/master-0/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786008495763054-20932-0/minicluster-data/master-0/wal
uuid: "e749c4a6f6294d4a800f80937d0413f9"
format_stamp: "Formatted at 2026-08-06 09:28:58 on dist-test-slave-45dx"
server_key: "64dd37dd1fff43a4e6705d624a7ffacd"
server_key_iv: "0a18cbb23ac7d76b20b8e42190bfc39b"
server_key_version: "encryptionkey@0"
I20260806 09:28:58.638245 27639 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786008495763054-20932-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786008495763054-20932-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786008495763054-20932-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:28:58.644068 27639 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:28:58.644887 27639 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:28:58.645190 27639 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:28:58.663784 27639 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.62:38975
I20260806 09:28:58.663826 27729 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.62:38975 every 8 connection(s)
I20260806 09:28:58.665130 27639 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
I20260806 09:28:58.668895 27730 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:28:58.673091 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 27639
I20260806 09:28:58.673295 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786008495763054-20932-0/minicluster-data/master-0/wal/instance
I20260806 09:28:58.673954 20932 external_mini_cluster.cc:1428] Setting key 4ef71df735d5698ecc5a77486055d0e7
I20260806 09:28:58.679827 27730 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e749c4a6f6294d4a800f80937d0413f9: Bootstrap starting.
I20260806 09:28:58.682247 27735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:58.675918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41236 (local address 127.20.113.62:38975)
0806 09:28:58.676971 (+ 1053us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:58.676982 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:58.677034 (+ 52us) server_negotiation.cc:407] Connection header received
0806 09:28:58.678737 (+ 1703us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:58.678759 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:58.679146 (+ 387us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:58.679453 (+ 307us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:28:58.680641 (+ 1188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:28:58.680646 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:28:58.680658 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:28:58.680671 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:28:58.680750 (+ 79us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:28:58.680765 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:28:58.680942 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:58.681037 (+ 95us) server_negotiation.cc:299] Negotiation successful
0806 09:28:58.681286 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":646,"thread_start_us":430,"threads_started":1}
I20260806 09:28:58.683444 27730 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e749c4a6f6294d4a800f80937d0413f9: Neither blocks nor log segments found. Creating new log.
I20260806 09:28:58.684861 27730 log.cc:824] T 00000000000000000000000000000000 P e749c4a6f6294d4a800f80937d0413f9: Log is configured to *not* fsync() on all Append() calls
I20260806 09:28:58.687548 27730 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e749c4a6f6294d4a800f80937d0413f9: No bootstrap required, opened a new log
I20260806 09:28:58.695005 27730 raft_consensus.cc:348] T 00000000000000000000000000000000 P e749c4a6f6294d4a800f80937d0413f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e749c4a6f6294d4a800f80937d0413f9" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 38975 } }
I20260806 09:28:58.695385 27730 raft_consensus.cc:374] T 00000000000000000000000000000000 P e749c4a6f6294d4a800f80937d0413f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:28:58.695497 27730 raft_consensus.cc:729] T 00000000000000000000000000000000 P e749c4a6f6294d4a800f80937d0413f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e749c4a6f6294d4a800f80937d0413f9, State: Initialized, Role: FOLLOWER
I20260806 09:28:58.696098 27730 consensus_queue.cc:260] T 00000000000000000000000000000000 P e749c4a6f6294d4a800f80937d0413f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e749c4a6f6294d4a800f80937d0413f9" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 38975 } }
I20260806 09:28:58.696368 27730 raft_consensus.cc:388] T 00000000000000000000000000000000 P e749c4a6f6294d4a800f80937d0413f9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 09:28:58.696542 27730 raft_consensus.cc:482] T 00000000000000000000000000000000 P e749c4a6f6294d4a800f80937d0413f9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 09:28:58.696717 27730 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e749c4a6f6294d4a800f80937d0413f9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:28:58.698849 27730 raft_consensus.cc:504] T 00000000000000000000000000000000 P e749c4a6f6294d4a800f80937d0413f9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e749c4a6f6294d4a800f80937d0413f9" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 38975 } }
I20260806 09:28:58.699318 27730 leader_election.cc:302] T 00000000000000000000000000000000 P e749c4a6f6294d4a800f80937d0413f9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e749c4a6f6294d4a800f80937d0413f9; no voters:
I20260806 09:28:58.699921 27730 leader_election.cc:288] T 00000000000000000000000000000000 P e749c4a6f6294d4a800f80937d0413f9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 09:28:58.700093 27742 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e749c4a6f6294d4a800f80937d0413f9 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:28:58.700878 27742 raft_consensus.cc:686] T 00000000000000000000000000000000 P e749c4a6f6294d4a800f80937d0413f9 [term 1 LEADER]: Becoming Leader. State: Replica: e749c4a6f6294d4a800f80937d0413f9, State: Running, Role: LEADER
I20260806 09:28:58.701476 27742 consensus_queue.cc:237] T 00000000000000000000000000000000 P e749c4a6f6294d4a800f80937d0413f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e749c4a6f6294d4a800f80937d0413f9" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 38975 } }
I20260806 09:28:58.702092 27730 sys_catalog.cc:564] T 00000000000000000000000000000000 P e749c4a6f6294d4a800f80937d0413f9 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 09:28:58.704561 27744 sys_catalog.cc:455] T 00000000000000000000000000000000 P e749c4a6f6294d4a800f80937d0413f9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e749c4a6f6294d4a800f80937d0413f9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e749c4a6f6294d4a800f80937d0413f9" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 38975 } } }
I20260806 09:28:58.704561 27743 sys_catalog.cc:455] T 00000000000000000000000000000000 P e749c4a6f6294d4a800f80937d0413f9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e749c4a6f6294d4a800f80937d0413f9. Latest consensus state: current_term: 1 leader_uuid: "e749c4a6f6294d4a800f80937d0413f9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e749c4a6f6294d4a800f80937d0413f9" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 38975 } } }
I20260806 09:28:58.705034 27744 sys_catalog.cc:458] T 00000000000000000000000000000000 P e749c4a6f6294d4a800f80937d0413f9 [sys.catalog]: This master's current role is: LEADER
I20260806 09:28:58.705092 27743 sys_catalog.cc:458] T 00000000000000000000000000000000 P e749c4a6f6294d4a800f80937d0413f9 [sys.catalog]: This master's current role is: LEADER
I20260806 09:28:58.706912 27752 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 09:28:58.711890 27752 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 09:28:58.720932 27752 catalog_manager.cc:1347] Generated new cluster ID: c95ec6ce20a541d78bc7550bf7c3e772
I20260806 09:28:58.721055 27752 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 09:28:58.755683 27752 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 09:28:58.756170 27752 catalog_manager.cc:1504] Loading token signing keys...
I20260806 09:28:58.762949 27752 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e749c4a6f6294d4a800f80937d0413f9: Generated new TSK 0
I20260806 09:28:58.763995 27752 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 09:28:58.781893 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786008495763054-20932-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786008495763054-20932-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786008495763054-20932-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786008495763054-20932-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.113.1:0
--local_ip_for_outbound_sockets=127.20.113.1
--webserver_interface=127.20.113.1
--webserver_port=0
--tserver_master_addrs=127.20.113.62:38975
--builtin_ntp_servers=127.20.113.20:34387
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 09:28:59.127701 27770 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:28:59.127978 27770 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:28:59.128135 27770 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:28:59.135243 27770 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:28:59.135342 27770 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:28:59.135479 27770 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.113.1
I20260806 09:28:59.144824 27770 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:34387
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786008495763054-20932-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786008495763054-20932-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.113.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786008495763054-20932-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.113.1
--webserver_port=0
--tserver_master_addrs=127.20.113.62:38975
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.113.1
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786008495763054-20932-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:28:59.146153 27770 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:28:59.148047 27770 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:28:59.161608 27776 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:28:59.161700 27779 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:28:59.161646 27783 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:28:59.162324 27770 server_base.cc:1034] running on GCE node
I20260806 09:28:59.162951 27770 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:28:59.163815 27770 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:28:59.165017 27770 hybrid_clock.cc:648] HybridClock initialized: now 1786008539164975 us; error 33 us; skew 500 ppm
I20260806 09:28:59.165339 27770 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 09:28:59.173934 27770 webserver.cc:460] Webserver started at http://127.20.113.1:42855/ using document root <none> and password file <none>
I20260806 09:28:59.174726 27770 fs_manager.cc:359] Metadata directory not provided
I20260806 09:28:59.174886 27770 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:28:59.175252 27770 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:28:59.178416 27770 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786008495763054-20932-0/minicluster-data/ts-0/data/instance:
uuid: "ec4e14d6b42741329577ab696f4b2be7"
format_stamp: "Formatted at 2026-08-06 09:28:59 on dist-test-slave-45dx"
server_key: "ab23f2a5715235fc3e7e27d0b7c597a0"
server_key_iv: "e534c335507a7bca7f56f5fc70d82ef0"
server_key_version: "encryptionkey@0"
I20260806 09:28:59.179263 27770 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786008495763054-20932-0/minicluster-data/ts-0/wal/instance:
uuid: "ec4e14d6b42741329577ab696f4b2be7"
format_stamp: "Formatted at 2026-08-06 09:28:59 on dist-test-slave-45dx"
server_key: "ab23f2a5715235fc3e7e27d0b7c597a0"
server_key_iv: "e534c335507a7bca7f56f5fc70d82ef0"
server_key_version: "encryptionkey@0"
I20260806 09:28:59.184868 27770 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260806 09:28:59.188503 27790 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:28:59.189823 27770 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 09:28:59.190034 27770 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786008495763054-20932-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786008495763054-20932-0/minicluster-data/ts-0/wal
uuid: "ec4e14d6b42741329577ab696f4b2be7"
format_stamp: "Formatted at 2026-08-06 09:28:59 on dist-test-slave-45dx"
server_key: "ab23f2a5715235fc3e7e27d0b7c597a0"
server_key_iv: "e534c335507a7bca7f56f5fc70d82ef0"
server_key_version: "encryptionkey@0"
I20260806 09:28:59.190253 27770 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786008495763054-20932-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786008495763054-20932-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786008495763054-20932-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:28:59.196223 27770 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:28:59.197036 27770 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:28:59.197309 27770 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:28:59.198195 27770 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:28:59.199613 27770 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:28:59.199750 27770 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:28:59.199879 27770 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:28:59.199976 27770 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:28:59.234467 27770 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.1:41463
I20260806 09:28:59.234522 27952 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.1:41463 every 8 connection(s)
I20260806 09:28:59.235803 27770 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786008495763054-20932-0/minicluster-data/ts-0/data/info.pb
I20260806 09:28:59.239323 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 27770
I20260806 09:28:59.239537 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786008495763054-20932-0/minicluster-data/ts-0/wal/instance
I20260806 09:28:59.240209 20932 external_mini_cluster.cc:1428] Setting key 8109d88f5b781fd614540dfa9defbd8a
I20260806 09:28:59.243513 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786008495763054-20932-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786008495763054-20932-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786008495763054-20932-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786008495763054-20932-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.113.2:0
--local_ip_for_outbound_sockets=127.20.113.2
--webserver_interface=127.20.113.2
--webserver_port=0
--tserver_master_addrs=127.20.113.62:38975
--builtin_ntp_servers=127.20.113.20:34387
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260806 09:28:59.245095 27958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:59.238437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.1:37473 (local address 127.20.113.62:38975)
0806 09:28:59.238902 (+ 465us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:59.238907 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:59.240241 (+ 1334us) server_negotiation.cc:407] Connection header received
0806 09:28:59.242522 (+ 2281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:59.242528 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:59.242591 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:59.242714 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:28:59.244159 (+ 1445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:28:59.244168 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:28:59.244175 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:28:59.244199 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:28:59.244256 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:28:59.244275 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:28:59.244434 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:59.244593 (+ 159us) server_negotiation.cc:299] Negotiation successful
0806 09:28:59.244672 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":233,"threads_started":1}
I20260806 09:28:59.245695 27959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:59.239044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:38975 (local address 127.20.113.1:37473)
0806 09:28:59.240055 (+ 1011us) negotiation.cc:107] Waiting for socket to connect
0806 09:28:59.240106 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 09:28:59.242195 (+ 2089us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:28:59.242923 (+ 728us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:28:59.242930 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:28:59.243062 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:28:59.243863 (+ 801us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 09:28:59.243865 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:28:59.243902 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 09:28:59.243984 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:28:59.244395 (+ 411us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:28:59.244404 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:28:59.244410 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 09:28:59.244623 (+ 213us) client_negotiation.cc:240] Negotiation successful
0806 09:28:59.244812 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":515,"thread_start_us":339,"threads_started":1}
I20260806 09:28:59.247225 27953 heartbeater.cc:347] Connected to a master server at 127.20.113.62:38975
I20260806 09:28:59.247557 27953 heartbeater.cc:464] Registering TS with master...
I20260806 09:28:59.248131 27953 heartbeater.cc:511] Master 127.20.113.62:38975 requested a full tablet report, sending...
I20260806 09:28:59.250218 27682 ts_manager.cc:194] Registered new tserver with Master: ec4e14d6b42741329577ab696f4b2be7 (127.20.113.1:41463)
W20260806 09:28:59.583585 27960 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:28:59.583902 27960 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:28:59.584053 27960 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:28:59.591262 27960 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:28:59.591356 27960 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:28:59.591518 27960 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.113.2
I20260806 09:28:59.600651 27960 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:34387
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786008495763054-20932-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786008495763054-20932-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.113.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786008495763054-20932-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.113.2
--webserver_port=0
--tserver_master_addrs=127.20.113.62:38975
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.113.2
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786008495763054-20932-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:28:59.602097 27960 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:28:59.604012 27960 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:28:59.617172 27969 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:28:59.617120 27967 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:28:59.617278 27971 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:28:59.618242 27960 server_base.cc:1034] running on GCE node
I20260806 09:28:59.618901 27960 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:28:59.619814 27960 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:28:59.621033 27960 hybrid_clock.cc:648] HybridClock initialized: now 1786008539620987 us; error 25 us; skew 500 ppm
I20260806 09:28:59.621357 27960 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 09:28:59.623643 27960 webserver.cc:460] Webserver started at http://127.20.113.2:45395/ using document root <none> and password file <none>
I20260806 09:28:59.624423 27960 fs_manager.cc:359] Metadata directory not provided
I20260806 09:28:59.624542 27960 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:28:59.624881 27960 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:28:59.627812 27960 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786008495763054-20932-0/minicluster-data/ts-1/data/instance:
uuid: "b6b8d4d1772f4b49af4e4aee57ee6828"
format_stamp: "Formatted at 2026-08-06 09:28:59 on dist-test-slave-45dx"
server_key: "09fe9d6fd3655d08ab4b0da093153292"
server_key_iv: "6ab1f8ed44581dd815b64b998c9eb168"
server_key_version: "encryptionkey@0"
I20260806 09:28:59.628702 27960 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786008495763054-20932-0/minicluster-data/ts-1/wal/instance:
uuid: "b6b8d4d1772f4b49af4e4aee57ee6828"
format_stamp: "Formatted at 2026-08-06 09:28:59 on dist-test-slave-45dx"
server_key: "09fe9d6fd3655d08ab4b0da093153292"
server_key_iv: "6ab1f8ed44581dd815b64b998c9eb168"
server_key_version: "encryptionkey@0"
I20260806 09:28:59.634145 27960 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 09:28:59.637753 27980 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:28:59.639052 27960 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 09:28:59.639236 27960 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786008495763054-20932-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786008495763054-20932-0/minicluster-data/ts-1/wal
uuid: "b6b8d4d1772f4b49af4e4aee57ee6828"
format_stamp: "Formatted at 2026-08-06 09:28:59 on dist-test-slave-45dx"
server_key: "09fe9d6fd3655d08ab4b0da093153292"
server_key_iv: "6ab1f8ed44581dd815b64b998c9eb168"
server_key_version: "encryptionkey@0"
I20260806 09:28:59.639470 27960 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786008495763054-20932-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786008495763054-20932-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786008495763054-20932-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:28:59.645673 27960 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:28:59.646513 27960 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:28:59.646806 27960 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:28:59.647742 27960 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:28:59.649456 27960 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:28:59.649586 27960 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:28:59.649717 27960 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:28:59.649808 27960 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:28:59.685287 27960 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.2:37589
I20260806 09:28:59.685379 28152 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.2:37589 every 8 connection(s)
I20260806 09:28:59.686643 27960 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786008495763054-20932-0/minicluster-data/ts-1/data/info.pb
I20260806 09:28:59.690549 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 27960
I20260806 09:28:59.690716 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786008495763054-20932-0/minicluster-data/ts-1/wal/instance
I20260806 09:28:59.691306 20932 external_mini_cluster.cc:1428] Setting key 23d4b745f94f77228161278ab93f18b8
I20260806 09:28:59.694306 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786008495763054-20932-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786008495763054-20932-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786008495763054-20932-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786008495763054-20932-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.113.3:0
--local_ip_for_outbound_sockets=127.20.113.3
--webserver_interface=127.20.113.3
--webserver_port=0
--tserver_master_addrs=127.20.113.62:38975
--builtin_ntp_servers=127.20.113.20:34387
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260806 09:28:59.695626 27958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:59.689347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.2:47043 (local address 127.20.113.62:38975)
0806 09:28:59.689521 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0806 09:28:59.689526 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:28:59.691161 (+ 1635us) server_negotiation.cc:407] Connection header received
0806 09:28:59.693150 (+ 1989us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:28:59.693156 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:28:59.693241 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:28:59.693358 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:28:59.694871 (+ 1513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:28:59.694878 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:28:59.694882 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:28:59.694901 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:28:59.694941 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:28:59.694957 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:28:59.695085 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0806 09:28:59.695279 (+ 194us) server_negotiation.cc:299] Negotiation successful
0806 09:28:59.695357 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 09:28:59.696753 28156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:28:59.689872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:38975 (local address 127.20.113.2:47043)
0806 09:28:59.690984 (+ 1112us) negotiation.cc:107] Waiting for socket to connect
0806 09:28:59.691031 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 09:28:59.692881 (+ 1850us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:28:59.693590 (+ 709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:28:59.693598 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:28:59.693738 (+ 140us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:28:59.694501 (+ 763us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 09:28:59.694505 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:28:59.694555 (+ 50us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 09:28:59.694695 (+ 140us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:28:59.695062 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:28:59.695074 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:28:59.695081 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 09:28:59.695364 (+ 283us) client_negotiation.cc:240] Negotiation successful
0806 09:28:59.695597 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":535,"thread_start_us":305,"threads_started":1}
I20260806 09:28:59.700350 28153 heartbeater.cc:347] Connected to a master server at 127.20.113.62:38975
I20260806 09:28:59.700881 28153 heartbeater.cc:464] Registering TS with master...
I20260806 09:28:59.702127 28153 heartbeater.cc:511] Master 127.20.113.62:38975 requested a full tablet report, sending...
I20260806 09:28:59.703990 27682 ts_manager.cc:194] Registered new tserver with Master: b6b8d4d1772f4b49af4e4aee57ee6828 (127.20.113.2:37589)
W20260806 09:29:00.036515 28157 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:29:00.036890 28157 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:29:00.037067 28157 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:29:00.044849 28157 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:29:00.044993 28157 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:29:00.045178 28157 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.113.3
I20260806 09:29:00.054407 28157 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:34387
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786008495763054-20932-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786008495763054-20932-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.113.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786008495763054-20932-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.113.3
--webserver_port=0
--tserver_master_addrs=127.20.113.62:38975
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.113.3
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786008495763054-20932-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:29:00.055786 28157 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:29:00.057658 28157 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:29:00.070905 28165 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:29:00.071003 28166 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:29:00.070912 28169 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:29:00.071920 28157 server_base.cc:1034] running on GCE node
I20260806 09:29:00.072568 28157 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:29:00.073482 28157 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:29:00.074685 28157 hybrid_clock.cc:648] HybridClock initialized: now 1786008540074613 us; error 69 us; skew 500 ppm
I20260806 09:29:00.075026 28157 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 09:29:00.077515 28157 webserver.cc:460] Webserver started at http://127.20.113.3:33405/ using document root <none> and password file <none>
I20260806 09:29:00.078269 28157 fs_manager.cc:359] Metadata directory not provided
I20260806 09:29:00.078367 28157 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:29:00.078764 28157 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:29:00.081768 28157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786008495763054-20932-0/minicluster-data/ts-2/data/instance:
uuid: "a5cae77ae9224c84856922d097fb896c"
format_stamp: "Formatted at 2026-08-06 09:29:00 on dist-test-slave-45dx"
server_key: "49bf8f739c1860924dc8a37b08e4a26b"
server_key_iv: "6a1d6192e2186e52b4be11b8d859b17d"
server_key_version: "encryptionkey@0"
I20260806 09:29:00.082620 28157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786008495763054-20932-0/minicluster-data/ts-2/wal/instance:
uuid: "a5cae77ae9224c84856922d097fb896c"
format_stamp: "Formatted at 2026-08-06 09:29:00 on dist-test-slave-45dx"
server_key: "49bf8f739c1860924dc8a37b08e4a26b"
server_key_iv: "6a1d6192e2186e52b4be11b8d859b17d"
server_key_version: "encryptionkey@0"
I20260806 09:29:00.087915 28157 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 09:29:00.091464 28177 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:29:00.092898 28157 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 09:29:00.093089 28157 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786008495763054-20932-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786008495763054-20932-0/minicluster-data/ts-2/wal
uuid: "a5cae77ae9224c84856922d097fb896c"
format_stamp: "Formatted at 2026-08-06 09:29:00 on dist-test-slave-45dx"
server_key: "49bf8f739c1860924dc8a37b08e4a26b"
server_key_iv: "6a1d6192e2186e52b4be11b8d859b17d"
server_key_version: "encryptionkey@0"
I20260806 09:29:00.093307 28157 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786008495763054-20932-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786008495763054-20932-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786008495763054-20932-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:29:00.099185 28157 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:29:00.100011 28157 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:29:00.100291 28157 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:29:00.101298 28157 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:29:00.102903 28157 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:29:00.102995 28157 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:29:00.103163 28157 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:29:00.103248 28157 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:29:00.138729 28157 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.3:41387
I20260806 09:29:00.138823 28335 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.3:41387 every 8 connection(s)
I20260806 09:29:00.140105 28157 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786008495763054-20932-0/minicluster-data/ts-2/data/info.pb
I20260806 09:29:00.142455 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 28157
I20260806 09:29:00.142632 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786008495763054-20932-0/minicluster-data/ts-2/wal/instance
I20260806 09:29:00.143288 20932 external_mini_cluster.cc:1428] Setting key 6395a559b6324ab867e2895122ce8841
I20260806 09:29:00.149659 27958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:00.143078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.3:43009 (local address 127.20.113.62:38975)
0806 09:29:00.143291 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:00.143298 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:00.145107 (+ 1809us) server_negotiation.cc:407] Connection header received
0806 09:29:00.147333 (+ 2226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:00.147339 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:00.147407 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:00.147501 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:00.148892 (+ 1391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:29:00.148898 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:29:00.148902 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:29:00.148921 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:29:00.148961 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:29:00.148972 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:29:00.149091 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:00.149307 (+ 216us) server_negotiation.cc:299] Negotiation successful
0806 09:29:00.149379 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 09:29:00.150578 28338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:00.143724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:38975 (local address 127.20.113.3:43009)
0806 09:29:00.144969 (+ 1245us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:00.145006 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:00.147076 (+ 2070us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:00.147731 (+ 655us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:00.147739 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:00.147823 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:29:00.148590 (+ 767us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 09:29:00.148593 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:29:00.148631 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 09:29:00.148715 (+ 84us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:29:00.149081 (+ 366us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:29:00.149093 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:29:00.149104 (+ 11us) client_negotiation.cc:769] Sending connection context
0806 09:29:00.149403 (+ 299us) client_negotiation.cc:240] Negotiation successful
0806 09:29:00.149652 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":604,"thread_start_us":356,"threads_started":1}
I20260806 09:29:00.151821 28336 heartbeater.cc:347] Connected to a master server at 127.20.113.62:38975
I20260806 09:29:00.152139 28336 heartbeater.cc:464] Registering TS with master...
I20260806 09:29:00.152781 28336 heartbeater.cc:511] Master 127.20.113.62:38975 requested a full tablet report, sending...
I20260806 09:29:00.154140 27682 ts_manager.cc:194] Registered new tserver with Master: a5cae77ae9224c84856922d097fb896c (127.20.113.3:41387)
I20260806 09:29:00.158717 20932 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 09:29:00.165858 27958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:00.163252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41252 (local address 127.20.113.62:38975)
0806 09:29:00.163417 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:00.163421 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:00.163943 (+ 522us) server_negotiation.cc:407] Connection header received
0806 09:29:00.164135 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:00.164141 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:00.164202 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:00.164304 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:00.165191 (+ 887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:29:00.165196 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:29:00.165200 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:29:00.165217 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:29:00.165254 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:29:00.165265 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:29:00.165390 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:00.165518 (+ 128us) server_negotiation.cc:299] Negotiation successful
0806 09:29:00.165596 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 09:29:00.169679 27682 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41252:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 09:29:00.177071 27682 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 09:29:00.194597 28357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:00.190701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58132 (local address 127.20.113.3:41387)
0806 09:29:00.191395 (+ 694us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:00.191402 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:00.191431 (+ 29us) server_negotiation.cc:407] Connection header received
0806 09:29:00.191486 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:00.191491 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:00.191657 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:00.191820 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:00.193334 (+ 1514us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:29:00.193346 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:29:00.193361 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:29:00.193386 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:29:00.193448 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:29:00.193473 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:29:00.193657 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:00.194009 (+ 352us) server_negotiation.cc:299] Negotiation successful
0806 09:29:00.194117 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":559,"thread_start_us":718,"threads_started":1}
I20260806 09:29:00.194712 28355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:00.189775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.3:41387 (local address 127.0.0.1:58132)
0806 09:29:00.190809 (+ 1034us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:00.190875 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:00.191010 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:00.191837 (+ 827us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:00.191845 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:00.191947 (+ 102us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:29:00.192977 (+ 1030us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 09:29:00.192982 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:29:00.193013 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 09:29:00.193091 (+ 78us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:29:00.193715 (+ 624us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:29:00.193727 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:29:00.193734 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 09:29:00.194006 (+ 272us) client_negotiation.cc:240] Negotiation successful
0806 09:29:00.194128 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":779,"thread_start_us":558,"threads_started":1}
I20260806 09:29:00.198105 28355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:00.189965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.1:41463 (local address 127.0.0.1:38194)
0806 09:29:00.195431 (+ 5466us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:00.195449 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:00.195543 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:00.196128 (+ 585us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:00.196133 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:00.196182 (+ 49us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:29:00.196869 (+ 687us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 09:29:00.196873 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:29:00.196890 (+ 17us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 09:29:00.196934 (+ 44us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:29:00.197594 (+ 660us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:29:00.197600 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:29:00.197605 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 09:29:00.197789 (+ 184us) client_negotiation.cc:240] Negotiation successful
0806 09:29:00.197843 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5355,"threads_started":1}
I20260806 09:29:00.198299 28358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:00.191242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38194 (local address 127.20.113.1:41463)
0806 09:29:00.191765 (+ 523us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:00.191776 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:00.195554 (+ 3778us) server_negotiation.cc:407] Connection header received
0806 09:29:00.195649 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:00.195659 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:00.195907 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:00.196091 (+ 184us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:00.197335 (+ 1244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:29:00.197346 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:29:00.197362 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:29:00.197383 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:29:00.197443 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:29:00.197467 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:29:00.197637 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:00.197713 (+ 76us) server_negotiation.cc:299] Negotiation successful
0806 09:29:00.197818 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":373,"threads_started":1}
I20260806 09:29:00.199309 28241 tablet_service.cc:1467] Processing CreateTablet for tablet 82948e90ef4644efb32a89ffd2fa61e5 (DEFAULT_TABLE table=test-table [id=f7fa38fbbf764473bf66a8d07e4f404d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:29:00.200814 28241 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 82948e90ef4644efb32a89ffd2fa61e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:29:00.202829 28355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:00.194127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.2:37589 (local address 127.0.0.1:48176)
0806 09:29:00.198797 (+ 4670us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:00.198814 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:00.198886 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:00.201438 (+ 2552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:00.201441 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:00.201459 (+ 18us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:29:00.202015 (+ 556us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 09:29:00.202017 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:29:00.202030 (+ 13us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 09:29:00.202065 (+ 35us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:29:00.202498 (+ 433us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:29:00.202505 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:29:00.202512 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 09:29:00.202600 (+ 88us) client_negotiation.cc:240] Negotiation successful
0806 09:29:00.202637 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4588,"thread_start_us":6355,"threads_started":1}
I20260806 09:29:00.203023 28360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:00.194683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48176 (local address 127.20.113.2:37589)
0806 09:29:00.200762 (+ 6079us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:00.200773 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:00.200803 (+ 30us) server_negotiation.cc:407] Connection header received
0806 09:29:00.200992 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:00.201005 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:00.201210 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:00.201365 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:00.202266 (+ 901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:29:00.202274 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:29:00.202285 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:29:00.202303 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:29:00.202356 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:29:00.202388 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:29:00.202541 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:00.202606 (+ 65us) server_negotiation.cc:299] Negotiation successful
0806 09:29:00.202700 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5911,"thread_start_us":6130,"threads_started":1}
I20260806 09:29:00.206995 28053 tablet_service.cc:1467] Processing CreateTablet for tablet 82948e90ef4644efb32a89ffd2fa61e5 (DEFAULT_TABLE table=test-table [id=f7fa38fbbf764473bf66a8d07e4f404d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:29:00.208721 28053 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 82948e90ef4644efb32a89ffd2fa61e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:29:00.213924 28362 tablet_bootstrap.cc:492] T 82948e90ef4644efb32a89ffd2fa61e5 P a5cae77ae9224c84856922d097fb896c: Bootstrap starting.
I20260806 09:29:00.214586 27851 tablet_service.cc:1467] Processing CreateTablet for tablet 82948e90ef4644efb32a89ffd2fa61e5 (DEFAULT_TABLE table=test-table [id=f7fa38fbbf764473bf66a8d07e4f404d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:29:00.216001 27851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 82948e90ef4644efb32a89ffd2fa61e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:29:00.217693 28362 tablet_bootstrap.cc:654] T 82948e90ef4644efb32a89ffd2fa61e5 P a5cae77ae9224c84856922d097fb896c: Neither blocks nor log segments found. Creating new log.
I20260806 09:29:00.218688 28362 log.cc:824] T 82948e90ef4644efb32a89ffd2fa61e5 P a5cae77ae9224c84856922d097fb896c: Log is configured to *not* fsync() on all Append() calls
I20260806 09:29:00.221223 28362 tablet_bootstrap.cc:492] T 82948e90ef4644efb32a89ffd2fa61e5 P a5cae77ae9224c84856922d097fb896c: No bootstrap required, opened a new log
I20260806 09:29:00.221494 28362 ts_tablet_manager.cc:1397] T 82948e90ef4644efb32a89ffd2fa61e5 P a5cae77ae9224c84856922d097fb896c: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260806 09:29:00.223969 28364 tablet_bootstrap.cc:492] T 82948e90ef4644efb32a89ffd2fa61e5 P b6b8d4d1772f4b49af4e4aee57ee6828: Bootstrap starting.
I20260806 09:29:00.227562 28364 tablet_bootstrap.cc:654] T 82948e90ef4644efb32a89ffd2fa61e5 P b6b8d4d1772f4b49af4e4aee57ee6828: Neither blocks nor log segments found. Creating new log.
I20260806 09:29:00.229094 28364 log.cc:824] T 82948e90ef4644efb32a89ffd2fa61e5 P b6b8d4d1772f4b49af4e4aee57ee6828: Log is configured to *not* fsync() on all Append() calls
I20260806 09:29:00.229341 28366 tablet_bootstrap.cc:492] T 82948e90ef4644efb32a89ffd2fa61e5 P ec4e14d6b42741329577ab696f4b2be7: Bootstrap starting.
I20260806 09:29:00.232827 28366 tablet_bootstrap.cc:654] T 82948e90ef4644efb32a89ffd2fa61e5 P ec4e14d6b42741329577ab696f4b2be7: Neither blocks nor log segments found. Creating new log.
I20260806 09:29:00.233562 28362 raft_consensus.cc:348] T 82948e90ef4644efb32a89ffd2fa61e5 P a5cae77ae9224c84856922d097fb896c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5cae77ae9224c84856922d097fb896c" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 41387 } } peers { permanent_uuid: "ec4e14d6b42741329577ab696f4b2be7" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 41463 } } peers { permanent_uuid: "b6b8d4d1772f4b49af4e4aee57ee6828" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 37589 } }
I20260806 09:29:00.234019 28362 raft_consensus.cc:374] T 82948e90ef4644efb32a89ffd2fa61e5 P a5cae77ae9224c84856922d097fb896c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:29:00.234050 28366 log.cc:824] T 82948e90ef4644efb32a89ffd2fa61e5 P ec4e14d6b42741329577ab696f4b2be7: Log is configured to *not* fsync() on all Append() calls
I20260806 09:29:00.234151 28362 raft_consensus.cc:729] T 82948e90ef4644efb32a89ffd2fa61e5 P a5cae77ae9224c84856922d097fb896c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5cae77ae9224c84856922d097fb896c, State: Initialized, Role: FOLLOWER
I20260806 09:29:00.234935 28362 consensus_queue.cc:260] T 82948e90ef4644efb32a89ffd2fa61e5 P a5cae77ae9224c84856922d097fb896c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5cae77ae9224c84856922d097fb896c" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 41387 } } peers { permanent_uuid: "ec4e14d6b42741329577ab696f4b2be7" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 41463 } } peers { permanent_uuid: "b6b8d4d1772f4b49af4e4aee57ee6828" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 37589 } }
I20260806 09:29:00.235634 28364 tablet_bootstrap.cc:492] T 82948e90ef4644efb32a89ffd2fa61e5 P b6b8d4d1772f4b49af4e4aee57ee6828: No bootstrap required, opened a new log
I20260806 09:29:00.236012 28364 ts_tablet_manager.cc:1397] T 82948e90ef4644efb32a89ffd2fa61e5 P b6b8d4d1772f4b49af4e4aee57ee6828: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.004s
I20260806 09:29:00.239890 28362 ts_tablet_manager.cc:1428] T 82948e90ef4644efb32a89ffd2fa61e5 P a5cae77ae9224c84856922d097fb896c: Time spent starting tablet: real 0.018s user 0.009s sys 0.008s
I20260806 09:29:00.240243 28336 heartbeater.cc:503] Master 127.20.113.62:38975 was elected leader, sending a full tablet report...
I20260806 09:29:00.247339 28364 raft_consensus.cc:348] T 82948e90ef4644efb32a89ffd2fa61e5 P b6b8d4d1772f4b49af4e4aee57ee6828 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5cae77ae9224c84856922d097fb896c" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 41387 } } peers { permanent_uuid: "ec4e14d6b42741329577ab696f4b2be7" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 41463 } } peers { permanent_uuid: "b6b8d4d1772f4b49af4e4aee57ee6828" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 37589 } }
I20260806 09:29:00.247829 28364 raft_consensus.cc:374] T 82948e90ef4644efb32a89ffd2fa61e5 P b6b8d4d1772f4b49af4e4aee57ee6828 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:29:00.247947 28364 raft_consensus.cc:729] T 82948e90ef4644efb32a89ffd2fa61e5 P b6b8d4d1772f4b49af4e4aee57ee6828 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6b8d4d1772f4b49af4e4aee57ee6828, State: Initialized, Role: FOLLOWER
I20260806 09:29:00.248589 28364 consensus_queue.cc:260] T 82948e90ef4644efb32a89ffd2fa61e5 P b6b8d4d1772f4b49af4e4aee57ee6828 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5cae77ae9224c84856922d097fb896c" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 41387 } } peers { permanent_uuid: "ec4e14d6b42741329577ab696f4b2be7" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 41463 } } peers { permanent_uuid: "b6b8d4d1772f4b49af4e4aee57ee6828" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 37589 } }
I20260806 09:29:00.249642 28153 heartbeater.cc:503] Master 127.20.113.62:38975 was elected leader, sending a full tablet report...
I20260806 09:29:00.250129 28364 ts_tablet_manager.cc:1428] T 82948e90ef4644efb32a89ffd2fa61e5 P b6b8d4d1772f4b49af4e4aee57ee6828: Time spent starting tablet: real 0.014s user 0.008s sys 0.004s
I20260806 09:29:00.250661 28366 tablet_bootstrap.cc:492] T 82948e90ef4644efb32a89ffd2fa61e5 P ec4e14d6b42741329577ab696f4b2be7: No bootstrap required, opened a new log
I20260806 09:29:00.250963 28366 ts_tablet_manager.cc:1397] T 82948e90ef4644efb32a89ffd2fa61e5 P ec4e14d6b42741329577ab696f4b2be7: Time spent bootstrapping tablet: real 0.022s user 0.015s sys 0.005s
I20260806 09:29:00.252947 27953 heartbeater.cc:503] Master 127.20.113.62:38975 was elected leader, sending a full tablet report...
I20260806 09:29:00.259356 28366 raft_consensus.cc:348] T 82948e90ef4644efb32a89ffd2fa61e5 P ec4e14d6b42741329577ab696f4b2be7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5cae77ae9224c84856922d097fb896c" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 41387 } } peers { permanent_uuid: "ec4e14d6b42741329577ab696f4b2be7" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 41463 } } peers { permanent_uuid: "b6b8d4d1772f4b49af4e4aee57ee6828" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 37589 } }
I20260806 09:29:00.259763 28366 raft_consensus.cc:374] T 82948e90ef4644efb32a89ffd2fa61e5 P ec4e14d6b42741329577ab696f4b2be7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:29:00.259865 28366 raft_consensus.cc:729] T 82948e90ef4644efb32a89ffd2fa61e5 P ec4e14d6b42741329577ab696f4b2be7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec4e14d6b42741329577ab696f4b2be7, State: Initialized, Role: FOLLOWER
I20260806 09:29:00.260507 28366 consensus_queue.cc:260] T 82948e90ef4644efb32a89ffd2fa61e5 P ec4e14d6b42741329577ab696f4b2be7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5cae77ae9224c84856922d097fb896c" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 41387 } } peers { permanent_uuid: "ec4e14d6b42741329577ab696f4b2be7" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 41463 } } peers { permanent_uuid: "b6b8d4d1772f4b49af4e4aee57ee6828" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 37589 } }
I20260806 09:29:00.261056 28369 raft_consensus.cc:482] T 82948e90ef4644efb32a89ffd2fa61e5 P a5cae77ae9224c84856922d097fb896c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 09:29:00.261364 28369 raft_consensus.cc:504] T 82948e90ef4644efb32a89ffd2fa61e5 P a5cae77ae9224c84856922d097fb896c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5cae77ae9224c84856922d097fb896c" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 41387 } } peers { permanent_uuid: "ec4e14d6b42741329577ab696f4b2be7" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 41463 } } peers { permanent_uuid: "b6b8d4d1772f4b49af4e4aee57ee6828" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 37589 } }
I20260806 09:29:00.262413 28366 ts_tablet_manager.cc:1428] T 82948e90ef4644efb32a89ffd2fa61e5 P ec4e14d6b42741329577ab696f4b2be7: Time spent starting tablet: real 0.011s user 0.011s sys 0.000s
I20260806 09:29:00.263430 28369 leader_election.cc:288] T 82948e90ef4644efb32a89ffd2fa61e5 P a5cae77ae9224c84856922d097fb896c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ec4e14d6b42741329577ab696f4b2be7 (127.20.113.1:41463), b6b8d4d1772f4b49af4e4aee57ee6828 (127.20.113.2:37589)
I20260806 09:29:00.267429 28338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:00.263770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.1:41463 (local address 127.20.113.3:49593)
0806 09:29:00.264151 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:00.264173 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:00.264277 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:00.265652 (+ 1375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:00.265659 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:00.265687 (+ 28us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:29:00.266478 (+ 791us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 09:29:00.266482 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:29:00.266502 (+ 20us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 09:29:00.266553 (+ 51us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:29:00.266943 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:29:00.266948 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:29:00.266952 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 09:29:00.267079 (+ 127us) client_negotiation.cc:240] Negotiation successful
0806 09:29:00.267143 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260}
I20260806 09:29:00.267976 28358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:00.263912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.3:49593 (local address 127.20.113.1:41463)
0806 09:29:00.264700 (+ 788us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:00.264706 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:00.264727 (+ 21us) server_negotiation.cc:407] Connection header received
0806 09:29:00.264785 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:00.264792 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:00.264861 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:00.264964 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:00.266710 (+ 1746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:29:00.266718 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:29:00.266723 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:29:00.266741 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:29:00.266780 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:29:00.266795 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:29:00.266900 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:00.267643 (+ 743us) server_negotiation.cc:299] Negotiation successful
0806 09:29:00.267713 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":653}
I20260806 09:29:00.268458 28372 raft_consensus.cc:482] T 82948e90ef4644efb32a89ffd2fa61e5 P b6b8d4d1772f4b49af4e4aee57ee6828 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 09:29:00.268822 28372 raft_consensus.cc:504] T 82948e90ef4644efb32a89ffd2fa61e5 P b6b8d4d1772f4b49af4e4aee57ee6828 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5cae77ae9224c84856922d097fb896c" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 41387 } } peers { permanent_uuid: "ec4e14d6b42741329577ab696f4b2be7" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 41463 } } peers { permanent_uuid: "b6b8d4d1772f4b49af4e4aee57ee6828" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 37589 } }
I20260806 09:29:00.269083 27881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82948e90ef4644efb32a89ffd2fa61e5" candidate_uuid: "a5cae77ae9224c84856922d097fb896c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec4e14d6b42741329577ab696f4b2be7" is_pre_election: true
I20260806 09:29:00.269377 28374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:00.264148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.2:37589 (local address 127.20.113.3:46039)
0806 09:29:00.266208 (+ 2060us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:00.266228 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:00.266406 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:00.266907 (+ 501us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:00.266913 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:00.266941 (+ 28us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:29:00.267745 (+ 804us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 09:29:00.267749 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:29:00.267763 (+ 14us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 09:29:00.267803 (+ 40us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:29:00.268267 (+ 464us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:29:00.268271 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:29:00.268275 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 09:29:00.268910 (+ 635us) client_negotiation.cc:240] Negotiation successful
0806 09:29:00.268966 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1940,"thread_start_us":1755,"threads_started":1}
I20260806 09:29:00.269680 27881 raft_consensus.cc:2454] T 82948e90ef4644efb32a89ffd2fa61e5 P ec4e14d6b42741329577ab696f4b2be7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a5cae77ae9224c84856922d097fb896c in term 0.
I20260806 09:29:00.270315 28181 leader_election.cc:302] T 82948e90ef4644efb32a89ffd2fa61e5 P a5cae77ae9224c84856922d097fb896c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a5cae77ae9224c84856922d097fb896c, ec4e14d6b42741329577ab696f4b2be7; no voters:
I20260806 09:29:00.270737 28369 raft_consensus.cc:2781] T 82948e90ef4644efb32a89ffd2fa61e5 P a5cae77ae9224c84856922d097fb896c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 09:29:00.270789 28372 leader_election.cc:288] T 82948e90ef4644efb32a89ffd2fa61e5 P b6b8d4d1772f4b49af4e4aee57ee6828 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a5cae77ae9224c84856922d097fb896c (127.20.113.3:41387), ec4e14d6b42741329577ab696f4b2be7 (127.20.113.1:41463)
I20260806 09:29:00.270910 28369 raft_consensus.cc:482] T 82948e90ef4644efb32a89ffd2fa61e5 P a5cae77ae9224c84856922d097fb896c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 09:29:00.271020 28369 raft_consensus.cc:3037] T 82948e90ef4644efb32a89ffd2fa61e5 P a5cae77ae9224c84856922d097fb896c [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:29:00.268831 28360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:00.265315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.3:46039 (local address 127.20.113.2:37589)
0806 09:29:00.265547 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:00.265553 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:00.266303 (+ 750us) server_negotiation.cc:407] Connection header received
0806 09:29:00.266557 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:00.266563 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:00.266646 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:00.266738 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:00.267979 (+ 1241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:29:00.267985 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:29:00.267993 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:29:00.268016 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:29:00.268057 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:29:00.268075 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:29:00.268177 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:00.268464 (+ 287us) server_negotiation.cc:299] Negotiation successful
0806 09:29:00.268528 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 09:29:00.274626 28357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:00.271279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.2:44275 (local address 127.20.113.3:41387)
0806 09:29:00.271471 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:00.271478 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:00.272213 (+ 735us) server_negotiation.cc:407] Connection header received
0806 09:29:00.272469 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:00.272473 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:00.272542 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:00.272619 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:00.273865 (+ 1246us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:29:00.273871 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:29:00.273875 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:29:00.273893 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:29:00.273927 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:29:00.273941 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:29:00.274024 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:00.274266 (+ 242us) server_negotiation.cc:299] Negotiation successful
0806 09:29:00.274326 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 09:29:00.274425 28085 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82948e90ef4644efb32a89ffd2fa61e5" candidate_uuid: "a5cae77ae9224c84856922d097fb896c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6b8d4d1772f4b49af4e4aee57ee6828" is_pre_election: true
I20260806 09:29:00.274488 28369 raft_consensus.cc:504] T 82948e90ef4644efb32a89ffd2fa61e5 P a5cae77ae9224c84856922d097fb896c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5cae77ae9224c84856922d097fb896c" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 41387 } } peers { permanent_uuid: "ec4e14d6b42741329577ab696f4b2be7" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 41463 } } peers { permanent_uuid: "b6b8d4d1772f4b49af4e4aee57ee6828" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 37589 } }
I20260806 09:29:00.275106 28375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:00.271367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.3:41387 (local address 127.20.113.2:44275)
0806 09:29:00.272103 (+ 736us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:00.272124 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:00.272334 (+ 210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:00.272840 (+ 506us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:00.272851 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:00.272882 (+ 31us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:29:00.273648 (+ 766us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 09:29:00.273652 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:29:00.273669 (+ 17us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 09:29:00.273715 (+ 46us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:29:00.274121 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:29:00.274126 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:29:00.274131 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 09:29:00.274602 (+ 471us) client_negotiation.cc:240] Negotiation successful
0806 09:29:00.274665 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":565,"thread_start_us":1124,"threads_started":1}
I20260806 09:29:00.274935 28085 raft_consensus.cc:2454] T 82948e90ef4644efb32a89ffd2fa61e5 P b6b8d4d1772f4b49af4e4aee57ee6828 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a5cae77ae9224c84856922d097fb896c in term 0.
I20260806 09:29:00.276062 27881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82948e90ef4644efb32a89ffd2fa61e5" candidate_uuid: "a5cae77ae9224c84856922d097fb896c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec4e14d6b42741329577ab696f4b2be7"
I20260806 09:29:00.276165 28085 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82948e90ef4644efb32a89ffd2fa61e5" candidate_uuid: "a5cae77ae9224c84856922d097fb896c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6b8d4d1772f4b49af4e4aee57ee6828"
I20260806 09:29:00.276271 27881 raft_consensus.cc:3037] T 82948e90ef4644efb32a89ffd2fa61e5 P ec4e14d6b42741329577ab696f4b2be7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:29:00.276377 28085 raft_consensus.cc:3037] T 82948e90ef4644efb32a89ffd2fa61e5 P b6b8d4d1772f4b49af4e4aee57ee6828 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:29:00.276237 28265 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82948e90ef4644efb32a89ffd2fa61e5" candidate_uuid: "b6b8d4d1772f4b49af4e4aee57ee6828" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5cae77ae9224c84856922d097fb896c" is_pre_election: true
I20260806 09:29:00.276743 28265 raft_consensus.cc:2379] T 82948e90ef4644efb32a89ffd2fa61e5 P a5cae77ae9224c84856922d097fb896c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b6b8d4d1772f4b49af4e4aee57ee6828 in current term 1: Already voted for candidate a5cae77ae9224c84856922d097fb896c in this term.
I20260806 09:29:00.277050 28358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:00.272097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.2:35091 (local address 127.20.113.1:41463)
0806 09:29:00.272246 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:00.272252 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:00.273229 (+ 977us) server_negotiation.cc:407] Connection header received
0806 09:29:00.273457 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:00.273463 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:00.273545 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:00.273642 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:00.276166 (+ 2524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:29:00.276171 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:29:00.276177 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:29:00.276197 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:29:00.276233 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:29:00.276248 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:29:00.276386 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:00.276667 (+ 281us) server_negotiation.cc:299] Negotiation successful
0806 09:29:00.276735 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 09:29:00.277741 28369 leader_election.cc:288] T 82948e90ef4644efb32a89ffd2fa61e5 P a5cae77ae9224c84856922d097fb896c [CANDIDATE]: Term 1 election: Requested vote from peers ec4e14d6b42741329577ab696f4b2be7 (127.20.113.1:41463), b6b8d4d1772f4b49af4e4aee57ee6828 (127.20.113.2:37589)
I20260806 09:29:00.277935 28376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:00.271949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.1:41463 (local address 127.20.113.2:35091)
0806 09:29:00.273116 (+ 1167us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:00.273139 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:00.273302 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:00.273818 (+ 516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:00.273825 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:00.273852 (+ 27us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:29:00.275926 (+ 2074us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 09:29:00.275929 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:29:00.275945 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 09:29:00.275990 (+ 45us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:29:00.276498 (+ 508us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:29:00.276504 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:29:00.276508 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 09:29:00.277325 (+ 817us) client_negotiation.cc:240] Negotiation successful
0806 09:29:00.277373 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1039,"thread_start_us":1531,"threads_started":1}
I20260806 09:29:00.278666 27880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82948e90ef4644efb32a89ffd2fa61e5" candidate_uuid: "b6b8d4d1772f4b49af4e4aee57ee6828" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec4e14d6b42741329577ab696f4b2be7" is_pre_election: true
I20260806 09:29:00.279551 28085 raft_consensus.cc:2454] T 82948e90ef4644efb32a89ffd2fa61e5 P b6b8d4d1772f4b49af4e4aee57ee6828 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a5cae77ae9224c84856922d097fb896c in term 1.
I20260806 09:29:00.279533 27881 raft_consensus.cc:2454] T 82948e90ef4644efb32a89ffd2fa61e5 P ec4e14d6b42741329577ab696f4b2be7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a5cae77ae9224c84856922d097fb896c in term 1.
I20260806 09:29:00.279573 27983 leader_election.cc:302] T 82948e90ef4644efb32a89ffd2fa61e5 P b6b8d4d1772f4b49af4e4aee57ee6828 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b6b8d4d1772f4b49af4e4aee57ee6828; no voters: a5cae77ae9224c84856922d097fb896c, ec4e14d6b42741329577ab696f4b2be7
I20260806 09:29:00.279986 28372 raft_consensus.cc:2726] T 82948e90ef4644efb32a89ffd2fa61e5 P b6b8d4d1772f4b49af4e4aee57ee6828 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 09:29:00.280377 28185 leader_election.cc:302] T 82948e90ef4644efb32a89ffd2fa61e5 P a5cae77ae9224c84856922d097fb896c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a5cae77ae9224c84856922d097fb896c, b6b8d4d1772f4b49af4e4aee57ee6828; no voters:
I20260806 09:29:00.280681 28369 raft_consensus.cc:2781] T 82948e90ef4644efb32a89ffd2fa61e5 P a5cae77ae9224c84856922d097fb896c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:29:00.281167 28369 raft_consensus.cc:686] T 82948e90ef4644efb32a89ffd2fa61e5 P a5cae77ae9224c84856922d097fb896c [term 1 LEADER]: Becoming Leader. State: Replica: a5cae77ae9224c84856922d097fb896c, State: Running, Role: LEADER
I20260806 09:29:00.281605 28369 consensus_queue.cc:237] T 82948e90ef4644efb32a89ffd2fa61e5 P a5cae77ae9224c84856922d097fb896c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5cae77ae9224c84856922d097fb896c" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 41387 } } peers { permanent_uuid: "ec4e14d6b42741329577ab696f4b2be7" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 41463 } } peers { permanent_uuid: "b6b8d4d1772f4b49af4e4aee57ee6828" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 37589 } }
I20260806 09:29:00.286453 27681 catalog_manager.cc:5495] T 82948e90ef4644efb32a89ffd2fa61e5 P a5cae77ae9224c84856922d097fb896c reported cstate change: term changed from 0 to 1, leader changed from <none> to a5cae77ae9224c84856922d097fb896c (127.20.113.3). New cstate: current_term: 1 leader_uuid: "a5cae77ae9224c84856922d097fb896c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5cae77ae9224c84856922d097fb896c" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 41387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec4e14d6b42741329577ab696f4b2be7" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b6b8d4d1772f4b49af4e4aee57ee6828" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 37589 } health_report { overall_health: UNKNOWN } } }
I20260806 09:29:00.303354 28357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:00.301128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58146 (local address 127.20.113.3:41387)
0806 09:29:00.301335 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:00.301340 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:00.301359 (+ 19us) server_negotiation.cc:407] Connection header received
0806 09:29:00.301540 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:00.301544 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:00.301642 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:00.301756 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:00.302660 (+ 904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:29:00.302664 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:29:00.302667 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:29:00.302683 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:29:00.302715 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:29:00.302727 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:29:00.302831 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:00.302963 (+ 132us) server_negotiation.cc:299] Negotiation successful
0806 09:29:00.303010 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260806 09:29:00.311417 27881 raft_consensus.cc:1261] T 82948e90ef4644efb32a89ffd2fa61e5 P ec4e14d6b42741329577ab696f4b2be7 [term 1 FOLLOWER]: Refusing update from remote peer a5cae77ae9224c84856922d097fb896c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 09:29:00.311473 28085 raft_consensus.cc:1261] T 82948e90ef4644efb32a89ffd2fa61e5 P b6b8d4d1772f4b49af4e4aee57ee6828 [term 1 FOLLOWER]: Refusing update from remote peer a5cae77ae9224c84856922d097fb896c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 09:29:00.312294 28377 consensus_queue.cc:1035] T 82948e90ef4644efb32a89ffd2fa61e5 P a5cae77ae9224c84856922d097fb896c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec4e14d6b42741329577ab696f4b2be7" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 09:29:00.312716 28369 consensus_queue.cc:1035] T 82948e90ef4644efb32a89ffd2fa61e5 P a5cae77ae9224c84856922d097fb896c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6b8d4d1772f4b49af4e4aee57ee6828" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 37589 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 09:29:00.338442 27682 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:41252:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 09:29:00.338723 27682 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:41252:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 09:29:00.342299 27682 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e749c4a6f6294d4a800f80937d0413f9: Sending DeleteTablet for 3 replicas of tablet 82948e90ef4644efb32a89ffd2fa61e5
I20260806 09:29:00.343449 28241 tablet_service.cc:1514] Processing DeleteTablet for tablet 82948e90ef4644efb32a89ffd2fa61e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:29:00 UTC) from {username='slave'} at 127.0.0.1:58132
I20260806 09:29:00.343469 27851 tablet_service.cc:1514] Processing DeleteTablet for tablet 82948e90ef4644efb32a89ffd2fa61e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:29:00 UTC) from {username='slave'} at 127.0.0.1:38194
I20260806 09:29:00.343732 28053 tablet_service.cc:1514] Processing DeleteTablet for tablet 82948e90ef4644efb32a89ffd2fa61e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:29:00 UTC) from {username='slave'} at 127.0.0.1:48176
I20260806 09:29:00.345583 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 27770
I20260806 09:29:00.354524 28400 tablet_replica.cc:330] T 82948e90ef4644efb32a89ffd2fa61e5 P ec4e14d6b42741329577ab696f4b2be7: stopping tablet replica
I20260806 09:29:00.355268 28400 raft_consensus.cc:2229] T 82948e90ef4644efb32a89ffd2fa61e5 P ec4e14d6b42741329577ab696f4b2be7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 09:29:00.356093 28399 tablet_replica.cc:330] T 82948e90ef4644efb32a89ffd2fa61e5 P a5cae77ae9224c84856922d097fb896c: stopping tablet replica
I20260806 09:29:00.356786 28400 raft_consensus.cc:2258] T 82948e90ef4644efb32a89ffd2fa61e5 P ec4e14d6b42741329577ab696f4b2be7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 09:29:00.357811 28358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:00.346957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38204 (local address 127.20.113.1:41463)
0806 09:29:00.347123 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:00.347129 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:00.352367 (+ 5238us) server_negotiation.cc:407] Connection header received
0806 09:29:00.353170 (+ 803us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:00.353176 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:00.353287 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:00.353393 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:00.356930 (+ 3537us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:29:00.356937 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:29:00.356942 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:29:00.356963 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:29:00.357002 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:29:00.357019 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:29:00.357116 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:00.357405 (+ 289us) server_negotiation.cc:299] Negotiation successful
0806 09:29:00.357478 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 09:29:00.358676 27942 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:29:00.358680 28402 tablet_replica.cc:330] T 82948e90ef4644efb32a89ffd2fa61e5 P b6b8d4d1772f4b49af4e4aee57ee6828: stopping tablet replica
I20260806 09:29:00.359090 28402 raft_consensus.cc:2229] T 82948e90ef4644efb32a89ffd2fa61e5 P b6b8d4d1772f4b49af4e4aee57ee6828 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 09:29:00.359581 28402 raft_consensus.cc:2258] T 82948e90ef4644efb32a89ffd2fa61e5 P b6b8d4d1772f4b49af4e4aee57ee6828 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 09:29:00.360579 28399 raft_consensus.cc:2229] T 82948e90ef4644efb32a89ffd2fa61e5 P a5cae77ae9224c84856922d097fb896c [term 1 LEADER]: Raft consensus shutting down.
I20260806 09:29:00.361455 28399 raft_consensus.cc:2258] T 82948e90ef4644efb32a89ffd2fa61e5 P a5cae77ae9224c84856922d097fb896c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 09:29:00.362243 28402 ts_tablet_manager.cc:1905] T 82948e90ef4644efb32a89ffd2fa61e5 P b6b8d4d1772f4b49af4e4aee57ee6828: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 09:29:00.363955 28399 ts_tablet_manager.cc:1905] T 82948e90ef4644efb32a89ffd2fa61e5 P a5cae77ae9224c84856922d097fb896c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 09:29:00.370119 28402 ts_tablet_manager.cc:1918] T 82948e90ef4644efb32a89ffd2fa61e5 P b6b8d4d1772f4b49af4e4aee57ee6828: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 09:29:00.370285 28402 log.cc:1196] T 82948e90ef4644efb32a89ffd2fa61e5 P b6b8d4d1772f4b49af4e4aee57ee6828: Deleting WAL directory at /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786008495763054-20932-0/minicluster-data/ts-1/wal/wals/82948e90ef4644efb32a89ffd2fa61e5
I20260806 09:29:00.370867 28402 ts_tablet_manager.cc:1939] T 82948e90ef4644efb32a89ffd2fa61e5 P b6b8d4d1772f4b49af4e4aee57ee6828: Deleting consensus metadata
I20260806 09:29:00.372817 27665 catalog_manager.cc:4841] TS b6b8d4d1772f4b49af4e4aee57ee6828 (127.20.113.2:37589): tablet 82948e90ef4644efb32a89ffd2fa61e5 (table test-table [id=f7fa38fbbf764473bf66a8d07e4f404d]) successfully deleted
I20260806 09:29:00.372995 28399 ts_tablet_manager.cc:1918] T 82948e90ef4644efb32a89ffd2fa61e5 P a5cae77ae9224c84856922d097fb896c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 09:29:00.373186 28399 log.cc:1196] T 82948e90ef4644efb32a89ffd2fa61e5 P a5cae77ae9224c84856922d097fb896c: Deleting WAL directory at /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786008495763054-20932-0/minicluster-data/ts-2/wal/wals/82948e90ef4644efb32a89ffd2fa61e5
I20260806 09:29:00.373828 28399 ts_tablet_manager.cc:1939] T 82948e90ef4644efb32a89ffd2fa61e5 P a5cae77ae9224c84856922d097fb896c: Deleting consensus metadata
I20260806 09:29:00.375591 27661 catalog_manager.cc:4841] TS a5cae77ae9224c84856922d097fb896c (127.20.113.3:41387): tablet 82948e90ef4644efb32a89ffd2fa61e5 (table test-table [id=f7fa38fbbf764473bf66a8d07e4f404d]) successfully deleted
I20260806 09:29:00.457340 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 27770
W20260806 09:29:00.488168 27663 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.20.113.1:41463 (error 108)
I20260806 09:29:00.488535 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 27960
I20260806 09:29:00.491000 28356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:00.490229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.1:41463 (local address 127.0.0.1:38210)
0806 09:29:00.490467 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:00.490678 (+ 211us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.1:41463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
W20260806 09:29:00.491321 27663 catalog_manager.cc:4584] TS ec4e14d6b42741329577ab696f4b2be7 (127.20.113.1:41463): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 82948e90ef4644efb32a89ffd2fa61e5: Network error: Client connection negotiation failed: client connection to 127.20.113.1:41463: connect: Connection refused (error 111)
I20260806 09:29:00.492257 28360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:00.489924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48182 (local address 127.20.113.2:37589)
0806 09:29:00.490138 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:00.490145 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:00.490164 (+ 19us) server_negotiation.cc:407] Connection header received
0806 09:29:00.490232 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:00.490237 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:00.490354 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:00.490458 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:00.491568 (+ 1110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:29:00.491576 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:29:00.491580 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:29:00.491598 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:29:00.491636 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:29:00.491650 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:29:00.491795 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:00.491900 (+ 105us) server_negotiation.cc:299] Negotiation successful
0806 09:29:00.491960 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 09:29:00.493116 28141 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:29:00.545073 28356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:00.544578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.1:41463 (local address 127.0.0.1:38214)
0806 09:29:00.544808 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:00.544911 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.1:41463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260806 09:29:00.546259 28356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:00.545865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.1:41463 (local address 127.0.0.1:38222)
0806 09:29:00.546019 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:00.546101 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.1:41463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260806 09:29:00.606956 28356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:00.606339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.1:41463 (local address 127.0.0.1:38226)
0806 09:29:00.606590 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:00.606713 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.1:41463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260806 09:29:00.608377 28356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:00.607842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.1:41463 (local address 127.0.0.1:38230)
0806 09:29:00.608050 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:00.608138 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.1:41463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260806 09:29:00.633036 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 27960
I20260806 09:29:00.658190 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 28157
I20260806 09:29:00.661422 28357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:00.659053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58150 (local address 127.20.113.3:41387)
0806 09:29:00.659236 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:00.659243 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:00.659350 (+ 107us) server_negotiation.cc:407] Connection header received
0806 09:29:00.659567 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:00.659573 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:00.659642 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:00.659767 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:00.660750 (+ 983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:29:00.660756 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:29:00.660761 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:29:00.660780 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:29:00.660821 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:29:00.660836 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:29:00.660979 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:00.661080 (+ 101us) server_negotiation.cc:299] Negotiation successful
0806 09:29:00.661139 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 09:29:00.662174 28327 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:29:00.690894 28356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:00.690375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.1:41463 (local address 127.0.0.1:38244)
0806 09:29:00.690607 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:00.690712 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.1:41463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260806 09:29:00.692159 28356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:00.691652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.1:41463 (local address 127.0.0.1:38256)
0806 09:29:00.691882 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:00.691984 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.1:41463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
I20260806 09:29:00.808591 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 28157
I20260806 09:29:00.838898 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 27639
I20260806 09:29:00.840003 27723 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:29:00.946405 28356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:00.945821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.1:41463 (local address 127.0.0.1:38260)
0806 09:29:00.946085 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:00.946206 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.1:41463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260806 09:29:00.946511 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 27639
2026-08-06T09:29:00Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (2782 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260806 09:29:00.998265 20932 mini_oidc.cc:152] Starting JWKS server
I20260806 09:29:01.001092 20932 webserver.cc:460] Webserver started at https://127.0.0.1:40025/ using document root <none> and password file <none>
I20260806 09:29:01.001441 20932 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 09:29:01.002079 20932 webserver.cc:460] Webserver started at http://127.0.0.1:43989/ using document root <none> and password file <none>
2026-08-06T09:29:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T09:29:01Z Disabled control of system clock
I20260806 09:29:01.039350 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786008495763054-20932-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786008495763054-20932-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786008495763054-20932-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.113.62:35243
--webserver_interface=127.20.113.62
--webserver_port=0
--builtin_ntp_servers=127.20.113.20:38405
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.113.62:35243
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40025/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786008495763054-20932-0/testchainca.cert with env {}
W20260806 09:29:01.374981 28415 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:29:01.375272 28415 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:29:01.375432 28415 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:29:01.382786 28415 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 09:29:01.382895 28415 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:29:01.382970 28415 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:29:01.383074 28415 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 09:29:01.383153 28415 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 09:29:01.383261 28415 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 09:29:01.383317 28415 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:40025/jwks.json
I20260806 09:29:01.393584 28415 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:38405
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786008495763054-20932-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786008495763054-20932-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.113.62:35243
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.113.62:35243
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40025/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.113.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786008495763054-20932-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786008495763054-20932-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:29:01.394953 28415 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:29:01.396306 28415 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:29:01.408541 28422 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:29:01.408581 28425 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:29:01.408694 28427 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:29:01.409696 28415 server_base.cc:1034] running on GCE node
I20260806 09:29:01.410504 28415 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:29:01.412055 28415 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:29:01.413273 28415 hybrid_clock.cc:648] HybridClock initialized: now 1786008541413228 us; error 35 us; skew 500 ppm
I20260806 09:29:01.413668 28415 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 09:29:01.416072 28415 webserver.cc:460] Webserver started at http://127.20.113.62:35207/ using document root <none> and password file <none>
I20260806 09:29:01.416842 28415 fs_manager.cc:359] Metadata directory not provided
I20260806 09:29:01.416973 28415 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:29:01.417364 28415 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:29:01.420426 28415 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786008495763054-20932-0/minicluster-data/master-0/data/instance:
uuid: "2db31a47769f4ca98475b99014662d26"
format_stamp: "Formatted at 2026-08-06 09:29:01 on dist-test-slave-45dx"
server_key: "1ed1e61c72eb157f80ae2c5b436aa7c5"
server_key_iv: "5d7f83d721508408f7ca7ef027453ed1"
server_key_version: "encryptionkey@0"
I20260806 09:29:01.421332 28415 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786008495763054-20932-0/minicluster-data/master-0/wal/instance:
uuid: "2db31a47769f4ca98475b99014662d26"
format_stamp: "Formatted at 2026-08-06 09:29:01 on dist-test-slave-45dx"
server_key: "1ed1e61c72eb157f80ae2c5b436aa7c5"
server_key_iv: "5d7f83d721508408f7ca7ef027453ed1"
server_key_version: "encryptionkey@0"
I20260806 09:29:01.426771 28415 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260806 09:29:01.430310 28433 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:29:01.431696 28415 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 09:29:01.431900 28415 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786008495763054-20932-0/minicluster-data/master-0/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786008495763054-20932-0/minicluster-data/master-0/wal
uuid: "2db31a47769f4ca98475b99014662d26"
format_stamp: "Formatted at 2026-08-06 09:29:01 on dist-test-slave-45dx"
server_key: "1ed1e61c72eb157f80ae2c5b436aa7c5"
server_key_iv: "5d7f83d721508408f7ca7ef027453ed1"
server_key_version: "encryptionkey@0"
I20260806 09:29:01.432142 28415 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786008495763054-20932-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786008495763054-20932-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786008495763054-20932-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:29:01.467428 28415 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:29:01.468386 28415 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:29:01.468726 28415 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:29:01.488387 28415 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.62:35243
I20260806 09:29:01.488479 28509 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.62:35243 every 8 connection(s)
I20260806 09:29:01.489794 28415 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
I20260806 09:29:01.492028 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 28415
I20260806 09:29:01.492259 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786008495763054-20932-0/minicluster-data/master-0/wal/instance
I20260806 09:29:01.492944 20932 external_mini_cluster.cc:1428] Setting key 34fbcc3658c13f55aa84067169408def
I20260806 09:29:01.494048 28510 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:29:01.506105 28510 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2db31a47769f4ca98475b99014662d26: Bootstrap starting.
I20260806 09:29:01.506495 28513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:01.494837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35398 (local address 127.20.113.62:35243)
0806 09:29:01.495791 (+ 954us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:01.495804 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:01.495853 (+ 49us) server_negotiation.cc:407] Connection header received
0806 09:29:01.497510 (+ 1657us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:01.497531 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:01.497991 (+ 460us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:01.498396 (+ 405us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:01.499077 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:01.500488 (+ 1411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:01.501879 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:01.502588 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:01.502782 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:01.504109 (+ 1327us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:29:01.504116 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:29:01.504128 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:29:01.504737 (+ 609us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:29:01.504826 (+ 89us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:29:01.504847 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:29:01.504974 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:01.505204 (+ 230us) server_negotiation.cc:299] Negotiation successful
0806 09:29:01.505536 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":556,"thread_start_us":358,"threads_started":1}
I20260806 09:29:01.509461 28510 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2db31a47769f4ca98475b99014662d26: Neither blocks nor log segments found. Creating new log.
I20260806 09:29:01.510396 20932 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 09:29:01.510695 28510 log.cc:824] T 00000000000000000000000000000000 P 2db31a47769f4ca98475b99014662d26: Log is configured to *not* fsync() on all Append() calls
I20260806 09:29:01.513823 28510 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2db31a47769f4ca98475b99014662d26: No bootstrap required, opened a new log
I20260806 09:29:01.521875 28510 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2db31a47769f4ca98475b99014662d26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2db31a47769f4ca98475b99014662d26" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 35243 } }
I20260806 09:29:01.522233 28510 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2db31a47769f4ca98475b99014662d26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:29:01.522320 28510 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2db31a47769f4ca98475b99014662d26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2db31a47769f4ca98475b99014662d26, State: Initialized, Role: FOLLOWER
I20260806 09:29:01.522954 28510 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2db31a47769f4ca98475b99014662d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2db31a47769f4ca98475b99014662d26" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 35243 } }
I20260806 09:29:01.523232 28510 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2db31a47769f4ca98475b99014662d26 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 09:29:01.523394 28510 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2db31a47769f4ca98475b99014662d26 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 09:29:01.523573 28510 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2db31a47769f4ca98475b99014662d26 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:29:01.525893 28510 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2db31a47769f4ca98475b99014662d26 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2db31a47769f4ca98475b99014662d26" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 35243 } }
I20260806 09:29:01.526415 28510 leader_election.cc:302] T 00000000000000000000000000000000 P 2db31a47769f4ca98475b99014662d26 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2db31a47769f4ca98475b99014662d26; no voters:
I20260806 09:29:01.527025 28510 leader_election.cc:288] T 00000000000000000000000000000000 P 2db31a47769f4ca98475b99014662d26 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 09:29:01.527213 28515 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2db31a47769f4ca98475b99014662d26 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:29:01.527966 28515 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2db31a47769f4ca98475b99014662d26 [term 1 LEADER]: Becoming Leader. State: Replica: 2db31a47769f4ca98475b99014662d26, State: Running, Role: LEADER
I20260806 09:29:01.528548 28515 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2db31a47769f4ca98475b99014662d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2db31a47769f4ca98475b99014662d26" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 35243 } }
I20260806 09:29:01.529188 28510 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2db31a47769f4ca98475b99014662d26 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 09:29:01.531579 28516 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2db31a47769f4ca98475b99014662d26 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2db31a47769f4ca98475b99014662d26" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2db31a47769f4ca98475b99014662d26" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 35243 } } }
I20260806 09:29:01.532100 28516 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2db31a47769f4ca98475b99014662d26 [sys.catalog]: This master's current role is: LEADER
I20260806 09:29:01.534138 28519 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2db31a47769f4ca98475b99014662d26 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2db31a47769f4ca98475b99014662d26. Latest consensus state: current_term: 1 leader_uuid: "2db31a47769f4ca98475b99014662d26" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2db31a47769f4ca98475b99014662d26" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 35243 } } }
I20260806 09:29:01.534865 28519 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2db31a47769f4ca98475b99014662d26 [sys.catalog]: This master's current role is: LEADER
I20260806 09:29:01.535404 28523 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 09:29:01.540441 28523 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 09:29:01.548287 28523 catalog_manager.cc:1347] Generated new cluster ID: bcb465e7b2be4cd88bac2775302d9014
I20260806 09:29:01.548430 28523 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 09:29:01.560715 28523 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 09:29:01.561991 28523 catalog_manager.cc:1504] Loading token signing keys...
I20260806 09:29:01.569239 28523 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2db31a47769f4ca98475b99014662d26: Generated new TSK 0
I20260806 09:29:01.570339 28523 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260806 09:29:01.620922 28559 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260806 09:29:01.621138 28559 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260806 09:29:01.623513 28513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:01.599677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35406 (local address 127.20.113.62:35243)
0806 09:29:01.599928 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:01.599937 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:01.600576 (+ 639us) server_negotiation.cc:407] Connection header received
0806 09:29:01.600913 (+ 337us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:01.600919 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:01.601018 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:01.601177 (+ 159us) server_negotiation.cc:226] Negotiated authn=JWT
0806 09:29:01.601784 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:01.603029 (+ 1245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:01.604504 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:01.605171 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:01.605322 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:01.605844 (+ 522us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 09:29:01.622798 (+ 16954us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40025/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0806 09:29:01.623174 (+ 376us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:35406: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40025/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":73}
W20260806 09:29:01.623662 28513 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:35406: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40025/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260806 09:29:01.625114 28557 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 09:29:01.599579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:35243 (local address 127.0.0.1:35406)
0806 09:29:01.600472 (+ 893us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:01.600489 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:01.600633 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:01.601205 (+ 572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:01.601210 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:01.601219 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0806 09:29:01.601557 (+ 338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:01.601565 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:01.603172 (+ 1607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:01.603176 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:01.604373 (+ 1197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:01.604385 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:01.605387 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:01.605392 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:01.605512 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:01.605570 (+ 58us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 09:29:01.623081 (+ 17511us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40025/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0806 09:29:01.623364 (+ 283us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.20.113.62:35243: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40025/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":688,"thread_start_us":556,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.20.113.62:35243: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40025/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260806 09:29:01.627161 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 28415
I20260806 09:29:01.628443 28505 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:29:01.722062 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 28415
2026-08-06T09:29:01Z chronyd exiting
I20260806 09:29:01.782078 20932 test_util.cc:175] -----------------------------------------------
I20260806 09:29:01.782263 20932 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786008495763054-20932-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (789 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260806 09:29:01.786248 20932 mini_oidc.cc:152] Starting JWKS server
I20260806 09:29:01.788616 20932 webserver.cc:460] Webserver started at https://127.0.0.1:43275/ using document root <none> and password file <none>
I20260806 09:29:01.788873 20932 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 09:29:01.789553 20932 webserver.cc:460] Webserver started at http://127.0.0.1:37229/ using document root <none> and password file <none>
2026-08-06T09:29:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T09:29:01Z Disabled control of system clock
I20260806 09:29:01.829349 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786008495763054-20932-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786008495763054-20932-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786008495763054-20932-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.113.62:44355
--webserver_interface=127.20.113.62
--webserver_port=0
--builtin_ntp_servers=127.20.113.20:41197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.113.62:44355
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43275/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786008495763054-20932-0/testchainca.cert with env {}
W20260806 09:29:02.162633 28569 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:29:02.162904 28569 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:29:02.163091 28569 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:29:02.170325 28569 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 09:29:02.170428 28569 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:29:02.170542 28569 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:29:02.170651 28569 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 09:29:02.170728 28569 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 09:29:02.170815 28569 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 09:29:02.170887 28569 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:43275/jwks.json
I20260806 09:29:02.180733 28569 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:41197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786008495763054-20932-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786008495763054-20932-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.113.62:44355
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.113.62:44355
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43275/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.113.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786008495763054-20932-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786008495763054-20932-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:29:02.182036 28569 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:29:02.183408 28569 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:29:02.195654 28576 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:29:02.195612 28579 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:29:02.195560 28577 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:29:02.196734 28569 server_base.cc:1034] running on GCE node
I20260806 09:29:02.197677 28569 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:29:02.199242 28569 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:29:02.200408 28569 hybrid_clock.cc:648] HybridClock initialized: now 1786008542200347 us; error 53 us; skew 500 ppm
I20260806 09:29:02.200778 28569 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 09:29:02.203426 28569 webserver.cc:460] Webserver started at http://127.20.113.62:37253/ using document root <none> and password file <none>
I20260806 09:29:02.204136 28569 fs_manager.cc:359] Metadata directory not provided
I20260806 09:29:02.204305 28569 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:29:02.204669 28569 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:29:02.207633 28569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786008495763054-20932-0/minicluster-data/master-0/data/instance:
uuid: "643732e525ca44e2a0a7818b3b90d9cc"
format_stamp: "Formatted at 2026-08-06 09:29:02 on dist-test-slave-45dx"
server_key: "168ed25a7c94c4f3df607e6b1c97de70"
server_key_iv: "572a100a11c4971fb052e7072bbb8065"
server_key_version: "encryptionkey@0"
I20260806 09:29:02.208570 28569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786008495763054-20932-0/minicluster-data/master-0/wal/instance:
uuid: "643732e525ca44e2a0a7818b3b90d9cc"
format_stamp: "Formatted at 2026-08-06 09:29:02 on dist-test-slave-45dx"
server_key: "168ed25a7c94c4f3df607e6b1c97de70"
server_key_iv: "572a100a11c4971fb052e7072bbb8065"
server_key_version: "encryptionkey@0"
I20260806 09:29:02.213793 28569 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 09:29:02.217259 28588 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:29:02.218695 28569 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 09:29:02.218899 28569 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786008495763054-20932-0/minicluster-data/master-0/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786008495763054-20932-0/minicluster-data/master-0/wal
uuid: "643732e525ca44e2a0a7818b3b90d9cc"
format_stamp: "Formatted at 2026-08-06 09:29:02 on dist-test-slave-45dx"
server_key: "168ed25a7c94c4f3df607e6b1c97de70"
server_key_iv: "572a100a11c4971fb052e7072bbb8065"
server_key_version: "encryptionkey@0"
I20260806 09:29:02.219144 28569 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786008495763054-20932-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786008495763054-20932-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786008495763054-20932-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:29:02.236243 28569 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:29:02.237152 28569 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:29:02.237463 28569 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:29:02.255359 28569 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.62:44355
I20260806 09:29:02.255370 28660 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.62:44355 every 8 connection(s)
I20260806 09:29:02.256774 28569 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
I20260806 09:29:02.260548 28661 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:29:02.261682 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 28569
I20260806 09:29:02.261904 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786008495763054-20932-0/minicluster-data/master-0/wal/instance
I20260806 09:29:02.262856 20932 external_mini_cluster.cc:1428] Setting key 3ca4f87056beeed9f54a544136bdf45a
I20260806 09:29:02.275883 28661 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 643732e525ca44e2a0a7818b3b90d9cc: Bootstrap starting.
I20260806 09:29:02.278012 28669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:02.265380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42854 (local address 127.20.113.62:44355)
0806 09:29:02.266737 (+ 1357us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:02.266751 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:02.266801 (+ 50us) server_negotiation.cc:407] Connection header received
0806 09:29:02.269426 (+ 2625us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:02.269455 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:02.270014 (+ 559us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:02.270422 (+ 408us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:02.270975 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:02.272221 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:02.273557 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:02.274235 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:02.274421 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:02.275672 (+ 1251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:29:02.275680 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:29:02.275692 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:29:02.276263 (+ 571us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:29:02.276413 (+ 150us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:29:02.276439 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:29:02.276554 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:02.276773 (+ 219us) server_negotiation.cc:299] Negotiation successful
0806 09:29:02.277083 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":830,"thread_start_us":938,"threads_started":1}
I20260806 09:29:02.279245 28661 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 643732e525ca44e2a0a7818b3b90d9cc: Neither blocks nor log segments found. Creating new log.
I20260806 09:29:02.280391 28661 log.cc:824] T 00000000000000000000000000000000 P 643732e525ca44e2a0a7818b3b90d9cc: Log is configured to *not* fsync() on all Append() calls
I20260806 09:29:02.281894 20932 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 09:29:02.283438 28661 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 643732e525ca44e2a0a7818b3b90d9cc: No bootstrap required, opened a new log
I20260806 09:29:02.291320 28661 raft_consensus.cc:348] T 00000000000000000000000000000000 P 643732e525ca44e2a0a7818b3b90d9cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "643732e525ca44e2a0a7818b3b90d9cc" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 44355 } }
I20260806 09:29:02.291698 28661 raft_consensus.cc:374] T 00000000000000000000000000000000 P 643732e525ca44e2a0a7818b3b90d9cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:29:02.291814 28661 raft_consensus.cc:729] T 00000000000000000000000000000000 P 643732e525ca44e2a0a7818b3b90d9cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 643732e525ca44e2a0a7818b3b90d9cc, State: Initialized, Role: FOLLOWER
I20260806 09:29:02.292479 28661 consensus_queue.cc:260] T 00000000000000000000000000000000 P 643732e525ca44e2a0a7818b3b90d9cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "643732e525ca44e2a0a7818b3b90d9cc" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 44355 } }
I20260806 09:29:02.292799 28661 raft_consensus.cc:388] T 00000000000000000000000000000000 P 643732e525ca44e2a0a7818b3b90d9cc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 09:29:02.292949 28661 raft_consensus.cc:482] T 00000000000000000000000000000000 P 643732e525ca44e2a0a7818b3b90d9cc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 09:29:02.293148 28661 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 643732e525ca44e2a0a7818b3b90d9cc [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:29:02.295328 28661 raft_consensus.cc:504] T 00000000000000000000000000000000 P 643732e525ca44e2a0a7818b3b90d9cc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "643732e525ca44e2a0a7818b3b90d9cc" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 44355 } }
I20260806 09:29:02.295874 28661 leader_election.cc:302] T 00000000000000000000000000000000 P 643732e525ca44e2a0a7818b3b90d9cc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 643732e525ca44e2a0a7818b3b90d9cc; no voters:
I20260806 09:29:02.296561 28661 leader_election.cc:288] T 00000000000000000000000000000000 P 643732e525ca44e2a0a7818b3b90d9cc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 09:29:02.296693 28673 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 643732e525ca44e2a0a7818b3b90d9cc [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:29:02.297917 28673 raft_consensus.cc:686] T 00000000000000000000000000000000 P 643732e525ca44e2a0a7818b3b90d9cc [term 1 LEADER]: Becoming Leader. State: Replica: 643732e525ca44e2a0a7818b3b90d9cc, State: Running, Role: LEADER
I20260806 09:29:02.298516 28673 consensus_queue.cc:237] T 00000000000000000000000000000000 P 643732e525ca44e2a0a7818b3b90d9cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "643732e525ca44e2a0a7818b3b90d9cc" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 44355 } }
I20260806 09:29:02.298898 28661 sys_catalog.cc:564] T 00000000000000000000000000000000 P 643732e525ca44e2a0a7818b3b90d9cc [sys.catalog]: configured and running, proceeding with master startup.
I20260806 09:29:02.301787 28674 sys_catalog.cc:455] T 00000000000000000000000000000000 P 643732e525ca44e2a0a7818b3b90d9cc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "643732e525ca44e2a0a7818b3b90d9cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "643732e525ca44e2a0a7818b3b90d9cc" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 44355 } } }
I20260806 09:29:02.301764 28675 sys_catalog.cc:455] T 00000000000000000000000000000000 P 643732e525ca44e2a0a7818b3b90d9cc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 643732e525ca44e2a0a7818b3b90d9cc. Latest consensus state: current_term: 1 leader_uuid: "643732e525ca44e2a0a7818b3b90d9cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "643732e525ca44e2a0a7818b3b90d9cc" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 44355 } } }
I20260806 09:29:02.302346 28674 sys_catalog.cc:458] T 00000000000000000000000000000000 P 643732e525ca44e2a0a7818b3b90d9cc [sys.catalog]: This master's current role is: LEADER
I20260806 09:29:02.302403 28675 sys_catalog.cc:458] T 00000000000000000000000000000000 P 643732e525ca44e2a0a7818b3b90d9cc [sys.catalog]: This master's current role is: LEADER
I20260806 09:29:02.304020 28684 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 09:29:02.309384 28684 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 09:29:02.318691 28684 catalog_manager.cc:1347] Generated new cluster ID: 11da3f2dee1c47eebd4615a301586f72
I20260806 09:29:02.318813 28684 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 09:29:02.341023 28684 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 09:29:02.342919 28684 catalog_manager.cc:1504] Loading token signing keys...
I20260806 09:29:02.352029 28684 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 643732e525ca44e2a0a7818b3b90d9cc: Generated new TSK 0
I20260806 09:29:02.353175 28684 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260806 09:29:02.392079 28705 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260806 09:29:02.392242 28705 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260806 09:29:02.393592 28669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:02.373149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42862 (local address 127.20.113.62:44355)
0806 09:29:02.373324 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:02.373329 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:02.373783 (+ 454us) server_negotiation.cc:407] Connection header received
0806 09:29:02.373982 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:02.373988 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:02.374094 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:02.374274 (+ 180us) server_negotiation.cc:226] Negotiated authn=JWT
0806 09:29:02.374750 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:02.375872 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:02.377341 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:02.378060 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:02.378203 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:02.378711 (+ 508us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 09:29:02.392861 (+ 14150us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43275/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0806 09:29:02.393249 (+ 388us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:42862: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43275/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":43}
W20260806 09:29:02.393610 28703 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 09:29:02.372985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44355 (local address 127.0.0.1:42862)
0806 09:29:02.373666 (+ 681us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:02.373683 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:02.373807 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:02.374285 (+ 478us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:02.374288 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:02.374312 (+ 24us) client_negotiation.cc:189] Negotiated authn=JWT
0806 09:29:02.374586 (+ 274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:02.374594 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:02.376026 (+ 1432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:02.376030 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:02.377186 (+ 1156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:02.377198 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:02.378242 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:02.378247 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:02.378398 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:02.378432 (+ 34us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 09:29:02.393094 (+ 14662us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43275/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0806 09:29:02.393319 (+ 225us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.20.113.62:44355: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43275/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":552,"thread_start_us":387,"threads_started":1}
W20260806 09:29:02.393718 28669 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:42862: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43275/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
I20260806 09:29:02.395205 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 28569
I20260806 09:29:02.396555 28654 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:29:02.496598 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 28569
2026-08-06T09:29:02Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (773 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260806 09:29:02.559758 20932 mini_oidc.cc:152] Starting JWKS server
I20260806 09:29:02.561724 20932 webserver.cc:460] Webserver started at https://127.0.0.1:32777/ using document root <none> and password file <none>
I20260806 09:29:02.561997 20932 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 09:29:02.562626 20932 webserver.cc:460] Webserver started at http://127.0.0.1:39315/ using document root <none> and password file <none>
2026-08-06T09:29:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T09:29:02Z Disabled control of system clock
I20260806 09:29:02.601267 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786008495763054-20932-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786008495763054-20932-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786008495763054-20932-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.113.62:44579
--webserver_interface=127.20.113.62
--webserver_port=0
--builtin_ntp_servers=127.20.113.20:46355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.113.62:44579
--enable_jwt_token_auth=true
--jwks_url=https://localhost:32777/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260806 09:29:02.938514 28715 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:29:02.938865 28715 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:29:02.939028 28715 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:29:02.946244 28715 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 09:29:02.946386 28715 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:29:02.946480 28715 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:29:02.946593 28715 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 09:29:02.946671 28715 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 09:29:02.946760 28715 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 09:29:02.946841 28715 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:32777/jwks.json
I20260806 09:29:02.955789 28715 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:46355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786008495763054-20932-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786008495763054-20932-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.113.62:44579
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.113.62:44579
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:32777/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.113.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786008495763054-20932-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:29:02.957118 28715 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:29:02.958829 28715 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:29:02.970851 28728 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:29:02.970808 28730 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:29:02.970898 28727 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:29:02.973687 28715 server_base.cc:1034] running on GCE node
I20260806 09:29:02.974470 28715 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:29:02.975839 28715 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:29:02.977083 28715 hybrid_clock.cc:648] HybridClock initialized: now 1786008542977034 us; error 63 us; skew 500 ppm
I20260806 09:29:02.977437 28715 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 09:29:02.979632 28715 webserver.cc:460] Webserver started at http://127.20.113.62:45071/ using document root <none> and password file <none>
I20260806 09:29:02.980393 28715 fs_manager.cc:359] Metadata directory not provided
I20260806 09:29:02.980527 28715 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:29:02.980860 28715 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:29:02.983857 28715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786008495763054-20932-0/minicluster-data/master-0/data/instance:
uuid: "3f5a01ec8ffe4e5aa7b37f0ef702c97e"
format_stamp: "Formatted at 2026-08-06 09:29:02 on dist-test-slave-45dx"
server_key: "2604f0fca358bc43c745f9939dc52e04"
server_key_iv: "5ac946f31ee0ee867bd4ced9b4c9f04e"
server_key_version: "encryptionkey@0"
I20260806 09:29:02.984743 28715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786008495763054-20932-0/minicluster-data/master-0/wal/instance:
uuid: "3f5a01ec8ffe4e5aa7b37f0ef702c97e"
format_stamp: "Formatted at 2026-08-06 09:29:02 on dist-test-slave-45dx"
server_key: "2604f0fca358bc43c745f9939dc52e04"
server_key_iv: "5ac946f31ee0ee867bd4ced9b4c9f04e"
server_key_version: "encryptionkey@0"
I20260806 09:29:02.990530 28715 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 09:29:02.994486 28743 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:29:02.995846 28715 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 09:29:02.996068 28715 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786008495763054-20932-0/minicluster-data/master-0/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786008495763054-20932-0/minicluster-data/master-0/wal
uuid: "3f5a01ec8ffe4e5aa7b37f0ef702c97e"
format_stamp: "Formatted at 2026-08-06 09:29:02 on dist-test-slave-45dx"
server_key: "2604f0fca358bc43c745f9939dc52e04"
server_key_iv: "5ac946f31ee0ee867bd4ced9b4c9f04e"
server_key_version: "encryptionkey@0"
I20260806 09:29:02.996305 28715 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786008495763054-20932-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786008495763054-20932-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786008495763054-20932-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:29:03.027348 28715 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:29:03.028220 28715 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:29:03.028578 28715 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:29:03.047802 28715 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.62:44579
I20260806 09:29:03.047825 28816 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.62:44579 every 8 connection(s)
I20260806 09:29:03.049242 28715 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
I20260806 09:29:03.053098 28817 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:29:03.054723 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 28715
I20260806 09:29:03.054953 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786008495763054-20932-0/minicluster-data/master-0/wal/instance
I20260806 09:29:03.055585 20932 external_mini_cluster.cc:1428] Setting key 0c2edad689729669ed6fd3b9b7ef042e
I20260806 09:29:03.068652 28822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:03.057439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36790 (local address 127.20.113.62:44579)
0806 09:29:03.058456 (+ 1017us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:03.058466 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:03.058508 (+ 42us) server_negotiation.cc:407] Connection header received
0806 09:29:03.060060 (+ 1552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:03.060081 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:03.060586 (+ 505us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:03.060977 (+ 391us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:03.061511 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:03.062866 (+ 1355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:03.064195 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:03.064927 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:03.065138 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:03.066378 (+ 1240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:29:03.066385 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:29:03.066397 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:29:03.066906 (+ 509us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:29:03.066975 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:29:03.066995 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:29:03.067133 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:03.067349 (+ 216us) server_negotiation.cc:299] Negotiation successful
0806 09:29:03.067653 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":604,"thread_start_us":379,"threads_started":1}
I20260806 09:29:03.069319 28817 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f5a01ec8ffe4e5aa7b37f0ef702c97e: Bootstrap starting.
I20260806 09:29:03.072429 20932 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 09:29:03.074011 28817 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3f5a01ec8ffe4e5aa7b37f0ef702c97e: Neither blocks nor log segments found. Creating new log.
I20260806 09:29:03.075677 28817 log.cc:824] T 00000000000000000000000000000000 P 3f5a01ec8ffe4e5aa7b37f0ef702c97e: Log is configured to *not* fsync() on all Append() calls
I20260806 09:29:03.078984 28817 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f5a01ec8ffe4e5aa7b37f0ef702c97e: No bootstrap required, opened a new log
I20260806 09:29:03.089272 28817 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3f5a01ec8ffe4e5aa7b37f0ef702c97e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f5a01ec8ffe4e5aa7b37f0ef702c97e" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 44579 } }
I20260806 09:29:03.089773 28817 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3f5a01ec8ffe4e5aa7b37f0ef702c97e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:29:03.089917 28817 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3f5a01ec8ffe4e5aa7b37f0ef702c97e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f5a01ec8ffe4e5aa7b37f0ef702c97e, State: Initialized, Role: FOLLOWER
I20260806 09:29:03.090689 28817 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3f5a01ec8ffe4e5aa7b37f0ef702c97e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f5a01ec8ffe4e5aa7b37f0ef702c97e" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 44579 } }
I20260806 09:29:03.091027 28817 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3f5a01ec8ffe4e5aa7b37f0ef702c97e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 09:29:03.091231 28817 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3f5a01ec8ffe4e5aa7b37f0ef702c97e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 09:29:03.091475 28817 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3f5a01ec8ffe4e5aa7b37f0ef702c97e [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:29:03.094475 28817 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3f5a01ec8ffe4e5aa7b37f0ef702c97e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f5a01ec8ffe4e5aa7b37f0ef702c97e" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 44579 } }
I20260806 09:29:03.095121 28817 leader_election.cc:302] T 00000000000000000000000000000000 P 3f5a01ec8ffe4e5aa7b37f0ef702c97e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3f5a01ec8ffe4e5aa7b37f0ef702c97e; no voters:
I20260806 09:29:03.095876 28817 leader_election.cc:288] T 00000000000000000000000000000000 P 3f5a01ec8ffe4e5aa7b37f0ef702c97e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 09:29:03.096041 28825 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3f5a01ec8ffe4e5aa7b37f0ef702c97e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:29:03.096825 28825 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3f5a01ec8ffe4e5aa7b37f0ef702c97e [term 1 LEADER]: Becoming Leader. State: Replica: 3f5a01ec8ffe4e5aa7b37f0ef702c97e, State: Running, Role: LEADER
I20260806 09:29:03.097404 28825 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3f5a01ec8ffe4e5aa7b37f0ef702c97e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f5a01ec8ffe4e5aa7b37f0ef702c97e" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 44579 } }
I20260806 09:29:03.098726 28817 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3f5a01ec8ffe4e5aa7b37f0ef702c97e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 09:29:03.100208 28829 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f5a01ec8ffe4e5aa7b37f0ef702c97e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3f5a01ec8ffe4e5aa7b37f0ef702c97e. Latest consensus state: current_term: 1 leader_uuid: "3f5a01ec8ffe4e5aa7b37f0ef702c97e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f5a01ec8ffe4e5aa7b37f0ef702c97e" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 44579 } } }
I20260806 09:29:03.100661 28829 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f5a01ec8ffe4e5aa7b37f0ef702c97e [sys.catalog]: This master's current role is: LEADER
I20260806 09:29:03.101487 28828 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f5a01ec8ffe4e5aa7b37f0ef702c97e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3f5a01ec8ffe4e5aa7b37f0ef702c97e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f5a01ec8ffe4e5aa7b37f0ef702c97e" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 44579 } } }
I20260806 09:29:03.101883 28828 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f5a01ec8ffe4e5aa7b37f0ef702c97e [sys.catalog]: This master's current role is: LEADER
I20260806 09:29:03.103047 28833 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 09:29:03.108217 28833 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 09:29:03.117141 28833 catalog_manager.cc:1347] Generated new cluster ID: 1f827969fc874fe0b9ef8d0e7d993747
I20260806 09:29:03.117260 28833 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 09:29:03.130003 28833 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 09:29:03.131250 28833 catalog_manager.cc:1504] Loading token signing keys...
I20260806 09:29:03.139504 28833 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3f5a01ec8ffe4e5aa7b37f0ef702c97e: Generated new TSK 0
I20260806 09:29:03.140276 28833 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 09:29:03.168109 28822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:03.160417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36802 (local address 127.20.113.62:44579)
0806 09:29:03.160579 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:03.160584 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:03.161088 (+ 504us) server_negotiation.cc:407] Connection header received
0806 09:29:03.161350 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:03.161354 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:03.161422 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:03.161610 (+ 188us) server_negotiation.cc:226] Negotiated authn=JWT
0806 09:29:03.162137 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:03.162946 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:03.164386 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:03.164875 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:03.165025 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:03.165535 (+ 510us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 09:29:03.167392 (+ 1857us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32777/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0806 09:29:03.167743 (+ 351us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:36802: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32777/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":51}
W20260806 09:29:03.168239 28861 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 09:29:03.160397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44579 (local address 127.0.0.1:36802)
0806 09:29:03.160981 (+ 584us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:03.160997 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:03.161163 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:03.161622 (+ 459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:03.161629 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:03.161641 (+ 12us) client_negotiation.cc:189] Negotiated authn=JWT
0806 09:29:03.161971 (+ 330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:03.161980 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:03.163098 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:03.163104 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:03.164185 (+ 1081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:03.164195 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:03.165085 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:03.165094 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:03.165241 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:03.165282 (+ 41us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 09:29:03.167582 (+ 2300us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32777/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0806 09:29:03.167821 (+ 239us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.20.113.62:44579: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32777/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":319,"threads_started":1}
W20260806 09:29:03.168259 28822 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:36802: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32777/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
I20260806 09:29:03.169375 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 28715
I20260806 09:29:03.170547 28811 generic_service.cc:196] Checking for leaks (request via RPC)
E20260806 09:29:03.180228 28863 webserver.cc:533] Webserver: sslize failed:
E20260806 09:29:03.180384 28863 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260806 09:29:03.259550 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 28715
2026-08-06T09:29:03Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (762 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-06T09:29:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T09:29:03Z Disabled control of system clock
I20260806 09:29:03.361462 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786008495763054-20932-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786008495763054-20932-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786008495763054-20932-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.113.62:33219
--webserver_interface=127.20.113.62
--webserver_port=0
--builtin_ntp_servers=127.20.113.20:38341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.113.62:33219
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786008495763054-20932-0/testchainca.cert with env {}
W20260806 09:29:03.705044 28873 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:29:03.705329 28873 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:29:03.705500 28873 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:29:03.712554 28873 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 09:29:03.712690 28873 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:29:03.712764 28873 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:29:03.712901 28873 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 09:29:03.712982 28873 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 09:29:03.713076 28873 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 09:29:03.713155 28873 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260806 09:29:03.722992 28873 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:38341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786008495763054-20932-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786008495763054-20932-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.113.62:33219
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.113.62:33219
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.113.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786008495763054-20932-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786008495763054-20932-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:29:03.724375 28873 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:29:03.725721 28873 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:29:03.737686 28881 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:29:03.737780 28885 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:29:03.737823 28880 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:29:03.738866 28873 server_base.cc:1034] running on GCE node
I20260806 09:29:03.739764 28873 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:29:03.741467 28873 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:29:03.742695 28873 hybrid_clock.cc:648] HybridClock initialized: now 1786008543742635 us; error 57 us; skew 500 ppm
I20260806 09:29:03.743048 28873 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 09:29:03.745353 28873 webserver.cc:460] Webserver started at http://127.20.113.62:44361/ using document root <none> and password file <none>
I20260806 09:29:03.746083 28873 fs_manager.cc:359] Metadata directory not provided
I20260806 09:29:03.746222 28873 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:29:03.746544 28873 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:29:03.749465 28873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786008495763054-20932-0/minicluster-data/master-0/data/instance:
uuid: "2760822e26ee416785ec92038cfd2acb"
format_stamp: "Formatted at 2026-08-06 09:29:03 on dist-test-slave-45dx"
server_key: "ce5f2bbf8b8c285e0e8ce27e3e231230"
server_key_iv: "22085649b3f5d66a6ee621d01ec472f6"
server_key_version: "encryptionkey@0"
I20260806 09:29:03.750269 28873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786008495763054-20932-0/minicluster-data/master-0/wal/instance:
uuid: "2760822e26ee416785ec92038cfd2acb"
format_stamp: "Formatted at 2026-08-06 09:29:03 on dist-test-slave-45dx"
server_key: "ce5f2bbf8b8c285e0e8ce27e3e231230"
server_key_iv: "22085649b3f5d66a6ee621d01ec472f6"
server_key_version: "encryptionkey@0"
I20260806 09:29:03.755288 28873 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260806 09:29:03.758638 28895 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:29:03.759966 28873 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 09:29:03.760152 28873 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786008495763054-20932-0/minicluster-data/master-0/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786008495763054-20932-0/minicluster-data/master-0/wal
uuid: "2760822e26ee416785ec92038cfd2acb"
format_stamp: "Formatted at 2026-08-06 09:29:03 on dist-test-slave-45dx"
server_key: "ce5f2bbf8b8c285e0e8ce27e3e231230"
server_key_iv: "22085649b3f5d66a6ee621d01ec472f6"
server_key_version: "encryptionkey@0"
I20260806 09:29:03.760406 28873 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786008495763054-20932-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786008495763054-20932-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786008495763054-20932-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:29:03.785756 28873 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:29:03.786640 28873 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:29:03.786962 28873 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:29:03.806131 28873 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.62:33219
I20260806 09:29:03.806150 28974 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.62:33219 every 8 connection(s)
I20260806 09:29:03.807569 28873 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
I20260806 09:29:03.811365 28976 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:29:03.814913 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 28873
I20260806 09:29:03.815132 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786008495763054-20932-0/minicluster-data/master-0/wal/instance
I20260806 09:29:03.815827 20932 external_mini_cluster.cc:1428] Setting key e4750195a1a6027424a6c8541409381a
I20260806 09:29:03.822889 28976 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2760822e26ee416785ec92038cfd2acb: Bootstrap starting.
I20260806 09:29:03.826483 28976 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2760822e26ee416785ec92038cfd2acb: Neither blocks nor log segments found. Creating new log.
I20260806 09:29:03.827543 28976 log.cc:824] T 00000000000000000000000000000000 P 2760822e26ee416785ec92038cfd2acb: Log is configured to *not* fsync() on all Append() calls
I20260806 09:29:03.830638 28976 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2760822e26ee416785ec92038cfd2acb: No bootstrap required, opened a new log
I20260806 09:29:03.830648 28979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:03.817837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56676 (local address 127.20.113.62:33219)
0806 09:29:03.818820 (+ 983us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:03.818870 (+ 50us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:03.818913 (+ 43us) server_negotiation.cc:407] Connection header received
0806 09:29:03.821126 (+ 2213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:03.821151 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:03.821583 (+ 432us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:03.821951 (+ 368us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:03.822594 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:03.823998 (+ 1404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:03.825861 (+ 1863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:03.826526 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:03.826722 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:03.828066 (+ 1344us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:29:03.828081 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:29:03.828105 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:29:03.828943 (+ 838us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:29:03.829049 (+ 106us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:29:03.829070 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:29:03.829195 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:03.829424 (+ 229us) server_negotiation.cc:299] Negotiation successful
0806 09:29:03.829730 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":590,"thread_start_us":358,"threads_started":1}
I20260806 09:29:03.834631 20932 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 09:29:03.839078 28976 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2760822e26ee416785ec92038cfd2acb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2760822e26ee416785ec92038cfd2acb" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 33219 } }
I20260806 09:29:03.839511 28976 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2760822e26ee416785ec92038cfd2acb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:29:03.839618 28976 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2760822e26ee416785ec92038cfd2acb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2760822e26ee416785ec92038cfd2acb, State: Initialized, Role: FOLLOWER
I20260806 09:29:03.840183 28976 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2760822e26ee416785ec92038cfd2acb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2760822e26ee416785ec92038cfd2acb" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 33219 } }
I20260806 09:29:03.840451 28976 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2760822e26ee416785ec92038cfd2acb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 09:29:03.840592 28976 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2760822e26ee416785ec92038cfd2acb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 09:29:03.840768 28976 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2760822e26ee416785ec92038cfd2acb [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:29:03.843189 28976 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2760822e26ee416785ec92038cfd2acb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2760822e26ee416785ec92038cfd2acb" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 33219 } }
I20260806 09:29:03.843685 28976 leader_election.cc:302] T 00000000000000000000000000000000 P 2760822e26ee416785ec92038cfd2acb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2760822e26ee416785ec92038cfd2acb; no voters:
I20260806 09:29:03.844383 28976 leader_election.cc:288] T 00000000000000000000000000000000 P 2760822e26ee416785ec92038cfd2acb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 09:29:03.844563 28983 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2760822e26ee416785ec92038cfd2acb [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:29:03.845402 28983 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2760822e26ee416785ec92038cfd2acb [term 1 LEADER]: Becoming Leader. State: Replica: 2760822e26ee416785ec92038cfd2acb, State: Running, Role: LEADER
I20260806 09:29:03.845913 28983 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2760822e26ee416785ec92038cfd2acb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2760822e26ee416785ec92038cfd2acb" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 33219 } }
I20260806 09:29:03.846545 28976 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2760822e26ee416785ec92038cfd2acb [sys.catalog]: configured and running, proceeding with master startup.
I20260806 09:29:03.849223 28984 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2760822e26ee416785ec92038cfd2acb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2760822e26ee416785ec92038cfd2acb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2760822e26ee416785ec92038cfd2acb" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 33219 } } }
I20260806 09:29:03.849244 28985 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2760822e26ee416785ec92038cfd2acb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2760822e26ee416785ec92038cfd2acb. Latest consensus state: current_term: 1 leader_uuid: "2760822e26ee416785ec92038cfd2acb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2760822e26ee416785ec92038cfd2acb" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 33219 } } }
I20260806 09:29:03.849630 28985 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2760822e26ee416785ec92038cfd2acb [sys.catalog]: This master's current role is: LEADER
I20260806 09:29:03.849630 28984 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2760822e26ee416785ec92038cfd2acb [sys.catalog]: This master's current role is: LEADER
I20260806 09:29:03.851282 28993 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 09:29:03.856892 28993 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 09:29:03.866923 28993 catalog_manager.cc:1347] Generated new cluster ID: 7911fe1f9e7e46e387e75357258587ec
I20260806 09:29:03.867038 28993 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 09:29:03.886621 28993 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 09:29:03.888394 28993 catalog_manager.cc:1504] Loading token signing keys...
I20260806 09:29:03.898351 28993 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2760822e26ee416785ec92038cfd2acb: Generated new TSK 0
I20260806 09:29:03.899394 28993 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 09:29:04.361099 28979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:03.916148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56692 (local address 127.20.113.62:33219)
0806 09:29:03.916413 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:03.916420 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:03.916773 (+ 353us) server_negotiation.cc:407] Connection header received
0806 09:29:03.917039 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:03.917043 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:03.917117 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:03.917232 (+ 115us) server_negotiation.cc:226] Negotiated authn=JWT
0806 09:29:03.917761 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:03.918474 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:03.920530 (+ 2056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:03.921002 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:03.921101 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:03.921652 (+ 551us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 09:29:04.360481 (+438829us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0806 09:29:04.360850 (+ 369us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:56692: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":70}
W20260806 09:29:04.361258 28979 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:56692: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260806 09:29:04.361532 29024 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 09:29:03.916033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:33219 (local address 127.0.0.1:56692)
0806 09:29:03.916663 (+ 630us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:03.916683 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:03.916842 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:03.917288 (+ 446us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:03.917295 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:03.917309 (+ 14us) client_negotiation.cc:189] Negotiated authn=JWT
0806 09:29:03.917608 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:03.917621 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:03.918619 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:03.918625 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:03.920379 (+ 1754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:03.920396 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:03.921179 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:03.921187 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:03.921377 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:03.921424 (+ 47us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 09:29:04.360862 (+439438us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0806 09:29:04.361135 (+ 273us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.20.113.62:33219: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":324,"threads_started":1}
I20260806 09:29:04.363154 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 28873
I20260806 09:29:04.364360 28968 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:29:04.470034 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 28873
2026-08-06T09:29:04Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (1194 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (438 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (432 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786008495763054-20932-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 09:29:05 dist-test-slave-45dx krb5kdc[29044](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 09:29:05 dist-test-slave-45dx krb5kdc[29044](info): set up 2 sockets
Aug 06 09:29:05 dist-test-slave-45dx krb5kdc[29044](info): commencing operation
krb5kdc: starting...
W20260806 09:29:06.630461 20932 mini_kdc.cc:121] Time spent starting KDC: real 1.241s user 0.000s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786008495763054-20932-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786008495763054-20932-0/krb5kdc/test-user.keytab.
Aug 06 09:29:06 dist-test-slave-45dx krb5kdc[29044](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008546, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T09:29:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T09:29:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.113.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.113.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.113.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab.
Entry for principal kudu/127.20.113.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab.
I20260806 09:29:06.912077 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786008495763054-20932-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786008495763054-20932-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786008495763054-20932-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.113.62:37355
--webserver_interface=127.20.113.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab
--principal=kudu/127.20.113.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:35743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.113.62:37355
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:29:07.253917 29062 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:29:07.254235 29062 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:29:07.254325 29062 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:29:07.262117 29062 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 09:29:07.262234 29062 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:29:07.262372 29062 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:29:07.262470 29062 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 09:29:07.262558 29062 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 09:29:07.272707 29062 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:35743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786008495763054-20932-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786008495763054-20932-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.113.62:37355
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab
--principal=kudu/127.20.113.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.113.62:37355
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.113.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786008495763054-20932-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:29:07.274039 29062 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:29:07.275386 29062 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:29:07.287070 29069 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:29:07.287039 29075 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:29:07.287119 29070 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:29:07.288679 29062 server_base.cc:1034] running on GCE node
I20260806 09:29:07.289543 29062 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:29:07.290831 29062 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:29:07.292075 29062 hybrid_clock.cc:648] HybridClock initialized: now 1786008547292034 us; error 45 us; skew 500 ppm
I20260806 09:29:07.292512 29062 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:29:07 dist-test-slave-45dx krb5kdc[29044](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008547, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:29:07.295776 29062 init.cc:377] Logged in from keytab as kudu/127.20.113.62@KRBTEST.COM (short username kudu)
I20260806 09:29:07.297443 29062 webserver.cc:460] Webserver started at http://127.20.113.62:44793/ using document root <none> and password file <none>
I20260806 09:29:07.298189 29062 fs_manager.cc:359] Metadata directory not provided
I20260806 09:29:07.298353 29062 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:29:07.298697 29062 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:29:07.302023 29062 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786008495763054-20932-0/minicluster-data/master-0/data/instance:
uuid: "3c34ee17d218435c87a6ef5935520c49"
format_stamp: "Formatted at 2026-08-06 09:29:07 on dist-test-slave-45dx"
server_key: "650b25ae1efa008cfd2b4c6c0de2fc1b"
server_key_iv: "09868acd7b3780223f046c463fd4dcf5"
server_key_version: "encryptionkey@0"
I20260806 09:29:07.302898 29062 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786008495763054-20932-0/minicluster-data/master-0/wal/instance:
uuid: "3c34ee17d218435c87a6ef5935520c49"
format_stamp: "Formatted at 2026-08-06 09:29:07 on dist-test-slave-45dx"
server_key: "650b25ae1efa008cfd2b4c6c0de2fc1b"
server_key_iv: "09868acd7b3780223f046c463fd4dcf5"
server_key_version: "encryptionkey@0"
I20260806 09:29:07.308406 29062 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260806 09:29:07.312217 29084 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:29:07.313668 29062 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 09:29:07.313882 29062 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786008495763054-20932-0/minicluster-data/master-0/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786008495763054-20932-0/minicluster-data/master-0/wal
uuid: "3c34ee17d218435c87a6ef5935520c49"
format_stamp: "Formatted at 2026-08-06 09:29:07 on dist-test-slave-45dx"
server_key: "650b25ae1efa008cfd2b4c6c0de2fc1b"
server_key_iv: "09868acd7b3780223f046c463fd4dcf5"
server_key_version: "encryptionkey@0"
I20260806 09:29:07.314107 29062 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786008495763054-20932-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786008495763054-20932-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786008495763054-20932-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:29:07.326341 29062 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:29:07.330317 29062 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:29:07.330590 29062 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:29:07.348730 29062 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.62:37355
I20260806 09:29:07.348771 29165 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.62:37355 every 8 connection(s)
I20260806 09:29:07.350091 29062 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
I20260806 09:29:07.351060 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 29062
I20260806 09:29:07.351279 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786008495763054-20932-0/minicluster-data/master-0/wal/instance
I20260806 09:29:07.351869 20932 external_mini_cluster.cc:1428] Setting key 4f210f8434d02aa6d701664627c8d631
I20260806 09:29:07.354403 29167 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 06 09:29:07 dist-test-slave-45dx krb5kdc[29044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008546, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.113.62@KRBTEST.COM
I20260806 09:29:07.367039 29167 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c34ee17d218435c87a6ef5935520c49: Bootstrap starting.
I20260806 09:29:07.370453 29167 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3c34ee17d218435c87a6ef5935520c49: Neither blocks nor log segments found. Creating new log.
I20260806 09:29:07.371609 29167 log.cc:824] T 00000000000000000000000000000000 P 3c34ee17d218435c87a6ef5935520c49: Log is configured to *not* fsync() on all Append() calls
I20260806 09:29:07.375118 29167 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c34ee17d218435c87a6ef5935520c49: No bootstrap required, opened a new log
I20260806 09:29:07.375826 29170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:07.353883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42514 (local address 127.20.113.62:37355)
0806 09:29:07.354669 (+ 786us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:07.354686 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:07.354741 (+ 55us) server_negotiation.cc:407] Connection header received
0806 09:29:07.357107 (+ 2366us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:07.357134 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:07.357606 (+ 472us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:07.358052 (+ 446us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:07.359332 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:07.361256 (+ 1924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:07.362496 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:07.363547 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:07.363740 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:07.368002 (+ 4262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:29:07.368009 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:29:07.368022 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:29:07.368061 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:29:07.371322 (+ 3261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:07.372256 (+ 934us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:07.372272 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:07.372287 (+ 15us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:07.372576 (+ 289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:07.373000 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:07.373006 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:07.373009 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:07.373587 (+ 578us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:29:07.373803 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:07.374315 (+ 512us) server_negotiation.cc:299] Negotiation successful
0806 09:29:07.374672 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":565,"thread_start_us":318,"threads_started":1}
I20260806 09:29:07.383626 29167 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3c34ee17d218435c87a6ef5935520c49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c34ee17d218435c87a6ef5935520c49" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 37355 } }
I20260806 09:29:07.384008 29167 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3c34ee17d218435c87a6ef5935520c49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:29:07.384147 29167 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3c34ee17d218435c87a6ef5935520c49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c34ee17d218435c87a6ef5935520c49, State: Initialized, Role: FOLLOWER
I20260806 09:29:07.384805 29167 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3c34ee17d218435c87a6ef5935520c49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c34ee17d218435c87a6ef5935520c49" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 37355 } }
I20260806 09:29:07.385071 29167 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3c34ee17d218435c87a6ef5935520c49 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 09:29:07.385234 29167 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3c34ee17d218435c87a6ef5935520c49 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 09:29:07.385454 29167 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3c34ee17d218435c87a6ef5935520c49 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:29:07.387657 29167 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3c34ee17d218435c87a6ef5935520c49 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c34ee17d218435c87a6ef5935520c49" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 37355 } }
I20260806 09:29:07.388198 29167 leader_election.cc:302] T 00000000000000000000000000000000 P 3c34ee17d218435c87a6ef5935520c49 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3c34ee17d218435c87a6ef5935520c49; no voters:
I20260806 09:29:07.388815 29167 leader_election.cc:288] T 00000000000000000000000000000000 P 3c34ee17d218435c87a6ef5935520c49 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 09:29:07.388984 29175 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3c34ee17d218435c87a6ef5935520c49 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:29:07.389771 29175 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3c34ee17d218435c87a6ef5935520c49 [term 1 LEADER]: Becoming Leader. State: Replica: 3c34ee17d218435c87a6ef5935520c49, State: Running, Role: LEADER
I20260806 09:29:07.390338 29175 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3c34ee17d218435c87a6ef5935520c49 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c34ee17d218435c87a6ef5935520c49" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 37355 } }
I20260806 09:29:07.391098 29167 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3c34ee17d218435c87a6ef5935520c49 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 09:29:07.394461 29180 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c34ee17d218435c87a6ef5935520c49 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3c34ee17d218435c87a6ef5935520c49. Latest consensus state: current_term: 1 leader_uuid: "3c34ee17d218435c87a6ef5935520c49" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c34ee17d218435c87a6ef5935520c49" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 37355 } } }
I20260806 09:29:07.394526 29176 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c34ee17d218435c87a6ef5935520c49 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3c34ee17d218435c87a6ef5935520c49" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c34ee17d218435c87a6ef5935520c49" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 37355 } } }
I20260806 09:29:07.394898 29180 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c34ee17d218435c87a6ef5935520c49 [sys.catalog]: This master's current role is: LEADER
I20260806 09:29:07.394903 29176 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c34ee17d218435c87a6ef5935520c49 [sys.catalog]: This master's current role is: LEADER
I20260806 09:29:07.399354 29189 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 09:29:07.404538 29189 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 09:29:07.412734 29189 catalog_manager.cc:1347] Generated new cluster ID: 2795ae4ac1f04884b7d56b30347451a1
I20260806 09:29:07.412838 29189 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 09:29:07.448891 29189 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 09:29:07.450933 29189 catalog_manager.cc:1504] Loading token signing keys...
I20260806 09:29:07.460101 29189 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3c34ee17d218435c87a6ef5935520c49: Generated new TSK 0
I20260806 09:29:07.461187 29189 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.113.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.113.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.113.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.1.keytab.
Entry for principal kudu/127.20.113.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.1.keytab.
I20260806 09:29:07.536194 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786008495763054-20932-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786008495763054-20932-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786008495763054-20932-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786008495763054-20932-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.113.1:0
--local_ip_for_outbound_sockets=127.20.113.1
--webserver_interface=127.20.113.1
--webserver_port=0
--tserver_master_addrs=127.20.113.62:37355
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.1.keytab
--principal=kudu/127.20.113.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:35743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:29:07.875463 29210 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:29:07.875771 29210 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:29:07.875947 29210 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:29:07.883231 29210 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:29:07.883369 29210 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:29:07.883536 29210 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.113.1
I20260806 09:29:07.893349 29210 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:35743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786008495763054-20932-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786008495763054-20932-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.1.keytab
--principal=kudu/127.20.113.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.113.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786008495763054-20932-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.113.1
--webserver_port=0
--tserver_master_addrs=127.20.113.62:37355
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.113.1
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786008495763054-20932-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:29:07.894688 29210 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:29:07.896025 29210 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:29:07.908972 29222 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:29:07.908931 29217 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:29:07.909158 29218 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:29:07.910199 29210 server_base.cc:1034] running on GCE node
I20260806 09:29:07.910816 29210 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:29:07.911815 29210 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:29:07.913061 29210 hybrid_clock.cc:648] HybridClock initialized: now 1786008547913026 us; error 56 us; skew 500 ppm
I20260806 09:29:07.913381 29210 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:29:07 dist-test-slave-45dx krb5kdc[29044](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008547, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:29:07.916816 29210 init.cc:377] Logged in from keytab as kudu/127.20.113.1@KRBTEST.COM (short username kudu)
I20260806 09:29:07.918577 29210 webserver.cc:460] Webserver started at http://127.20.113.1:43969/ using document root <none> and password file <none>
I20260806 09:29:07.919312 29210 fs_manager.cc:359] Metadata directory not provided
I20260806 09:29:07.919469 29210 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:29:07.919837 29210 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:29:07.923550 29210 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786008495763054-20932-0/minicluster-data/ts-0/data/instance:
uuid: "dd995f3690864e4b888e36346b741cc7"
format_stamp: "Formatted at 2026-08-06 09:29:07 on dist-test-slave-45dx"
server_key: "8c39afb337a580bc974483834c35dd83"
server_key_iv: "f57be00f2c845d54575515582b61bf28"
server_key_version: "encryptionkey@0"
I20260806 09:29:07.924431 29210 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786008495763054-20932-0/minicluster-data/ts-0/wal/instance:
uuid: "dd995f3690864e4b888e36346b741cc7"
format_stamp: "Formatted at 2026-08-06 09:29:07 on dist-test-slave-45dx"
server_key: "8c39afb337a580bc974483834c35dd83"
server_key_iv: "f57be00f2c845d54575515582b61bf28"
server_key_version: "encryptionkey@0"
I20260806 09:29:07.929641 29210 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260806 09:29:07.932981 29231 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:29:07.934245 29210 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260806 09:29:07.934427 29210 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786008495763054-20932-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786008495763054-20932-0/minicluster-data/ts-0/wal
uuid: "dd995f3690864e4b888e36346b741cc7"
format_stamp: "Formatted at 2026-08-06 09:29:07 on dist-test-slave-45dx"
server_key: "8c39afb337a580bc974483834c35dd83"
server_key_iv: "f57be00f2c845d54575515582b61bf28"
server_key_version: "encryptionkey@0"
I20260806 09:29:07.934638 29210 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786008495763054-20932-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786008495763054-20932-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786008495763054-20932-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:29:07.957275 29210 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:29:07.961218 29210 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:29:07.961521 29210 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:29:07.962572 29210 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:29:07.963903 29210 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:29:07.964015 29210 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:29:07.964170 29210 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:29:07.964254 29210 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:29:07.998149 29210 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.1:36637
I20260806 09:29:07.998235 29404 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.1:36637 every 8 connection(s)
I20260806 09:29:07.999418 29210 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786008495763054-20932-0/minicluster-data/ts-0/data/info.pb
I20260806 09:29:08.002530 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 29210
I20260806 09:29:08.002745 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786008495763054-20932-0/minicluster-data/ts-0/wal/instance
I20260806 09:29:08.003675 20932 external_mini_cluster.cc:1428] Setting key a61385991d8faa96bd6ea9a9661ff7a9
Aug 06 09:29:08 dist-test-slave-45dx krb5kdc[29044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008547, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.1@KRBTEST.COM for kudu/127.20.113.62@KRBTEST.COM
I20260806 09:29:08.021057 29407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:08.001920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.1:38689 (local address 127.20.113.62:37355)
0806 09:29:08.002434 (+ 514us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:08.002439 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:08.003496 (+ 1057us) server_negotiation.cc:407] Connection header received
0806 09:29:08.005856 (+ 2360us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:08.005862 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:08.005949 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:08.006070 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:08.007835 (+ 1765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:08.008919 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:08.010713 (+ 1794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:08.011116 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:08.011251 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:08.014984 (+ 3733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:29:08.014989 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:29:08.014992 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:29:08.015022 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:29:08.017478 (+ 2456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:08.018245 (+ 767us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:08.018254 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:08.018259 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:08.018426 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:08.019039 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:08.019047 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:08.019053 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:08.019432 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:29:08.019586 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:08.020255 (+ 669us) server_negotiation.cc:299] Negotiation successful
0806 09:29:08.020587 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":387,"thread_start_us":283,"threads_started":1}
I20260806 09:29:08.021715 29408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:08.002537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:37355 (local address 127.20.113.1:38689)
0806 09:29:08.003334 (+ 797us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:08.003372 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:08.005558 (+ 2186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:08.006256 (+ 698us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:08.006263 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:08.006775 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:29:08.007630 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:08.007655 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:08.009077 (+ 1422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:08.009082 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:08.010564 (+ 1482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:08.010576 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:08.011344 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:08.011353 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:08.011626 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:08.012371 (+ 745us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:29:08.012410 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:29:08.014766 (+ 2356us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:29:08.017662 (+ 2896us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:29:08.017670 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:29:08.017684 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:29:08.018125 (+ 441us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:29:08.018589 (+ 464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:29:08.018596 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:29:08.018600 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:29:08.018886 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:29:08.019562 (+ 676us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:29:08.019567 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:29:08.019873 (+ 306us) client_negotiation.cc:769] Sending connection context
0806 09:29:08.020106 (+ 233us) client_negotiation.cc:240] Negotiation successful
0806 09:29:08.020602 (+ 496us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":561,"thread_start_us":333,"threads_started":1}
I20260806 09:29:08.023362 29405 heartbeater.cc:347] Connected to a master server at 127.20.113.62:37355
I20260806 09:29:08.023685 29405 heartbeater.cc:464] Registering TS with master...
I20260806 09:29:08.024459 29405 heartbeater.cc:511] Master 127.20.113.62:37355 requested a full tablet report, sending...
I20260806 09:29:08.026547 29115 ts_manager.cc:194] Registered new tserver with Master: dd995f3690864e4b888e36346b741cc7 (127.20.113.1:36637)
I20260806 09:29:08.027972 29115 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.113.1@KRBTEST.COM'} at 127.20.113.1:38689
WARNING: no policy specified for kudu/127.20.113.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.113.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.113.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.2.keytab.
Entry for principal kudu/127.20.113.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.2.keytab.
I20260806 09:29:08.069010 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786008495763054-20932-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786008495763054-20932-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786008495763054-20932-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786008495763054-20932-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.113.2:0
--local_ip_for_outbound_sockets=127.20.113.2
--webserver_interface=127.20.113.2
--webserver_port=0
--tserver_master_addrs=127.20.113.62:37355
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.2.keytab
--principal=kudu/127.20.113.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:35743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:29:08.424731 29411 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:29:08.425076 29411 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:29:08.425257 29411 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:29:08.432473 29411 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:29:08.432610 29411 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:29:08.432809 29411 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.113.2
I20260806 09:29:08.442698 29411 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:35743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786008495763054-20932-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786008495763054-20932-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.2.keytab
--principal=kudu/127.20.113.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.113.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786008495763054-20932-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.113.2
--webserver_port=0
--tserver_master_addrs=127.20.113.62:37355
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.113.2
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786008495763054-20932-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:29:08.444034 29411 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:29:08.445986 29411 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:29:08.459888 29420 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:29:08.460117 29419 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:29:08.460605 29422 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:29:08.461789 29411 server_base.cc:1034] running on GCE node
I20260806 09:29:08.462427 29411 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:29:08.463379 29411 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:29:08.464629 29411 hybrid_clock.cc:648] HybridClock initialized: now 1786008548464578 us; error 43 us; skew 500 ppm
I20260806 09:29:08.464993 29411 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:29:08 dist-test-slave-45dx krb5kdc[29044](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008548, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:29:08.468252 29411 init.cc:377] Logged in from keytab as kudu/127.20.113.2@KRBTEST.COM (short username kudu)
I20260806 09:29:08.469800 29411 webserver.cc:460] Webserver started at http://127.20.113.2:45595/ using document root <none> and password file <none>
I20260806 09:29:08.470551 29411 fs_manager.cc:359] Metadata directory not provided
I20260806 09:29:08.470713 29411 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:29:08.471063 29411 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:29:08.474148 29411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786008495763054-20932-0/minicluster-data/ts-1/data/instance:
uuid: "b5c5746b8fea498986dd89ad7759e31b"
format_stamp: "Formatted at 2026-08-06 09:29:08 on dist-test-slave-45dx"
server_key: "c10770f33430251e3dd73d7c4d4e879f"
server_key_iv: "e47fe145d3f18f6c020439d17bd33638"
server_key_version: "encryptionkey@0"
I20260806 09:29:08.474977 29411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786008495763054-20932-0/minicluster-data/ts-1/wal/instance:
uuid: "b5c5746b8fea498986dd89ad7759e31b"
format_stamp: "Formatted at 2026-08-06 09:29:08 on dist-test-slave-45dx"
server_key: "c10770f33430251e3dd73d7c4d4e879f"
server_key_iv: "e47fe145d3f18f6c020439d17bd33638"
server_key_version: "encryptionkey@0"
I20260806 09:29:08.480352 29411 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 09:29:08.483940 29434 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:29:08.485411 29411 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260806 09:29:08.485550 29411 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786008495763054-20932-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786008495763054-20932-0/minicluster-data/ts-1/wal
uuid: "b5c5746b8fea498986dd89ad7759e31b"
format_stamp: "Formatted at 2026-08-06 09:29:08 on dist-test-slave-45dx"
server_key: "c10770f33430251e3dd73d7c4d4e879f"
server_key_iv: "e47fe145d3f18f6c020439d17bd33638"
server_key_version: "encryptionkey@0"
I20260806 09:29:08.485731 29411 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786008495763054-20932-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786008495763054-20932-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786008495763054-20932-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:29:08.497735 29411 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:29:08.501675 29411 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:29:08.501930 29411 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:29:08.502810 29411 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:29:08.504058 29411 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:29:08.504134 29411 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:29:08.504344 29411 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:29:08.504410 29411 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260806 09:29:08.539487 29411 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.2:42921
I20260806 09:29:08.539525 29603 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.2:42921 every 8 connection(s)
I20260806 09:29:08.540889 29411 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786008495763054-20932-0/minicluster-data/ts-1/data/info.pb
I20260806 09:29:08.548558 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 29411
I20260806 09:29:08.548746 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786008495763054-20932-0/minicluster-data/ts-1/wal/instance
I20260806 09:29:08.549214 20932 external_mini_cluster.cc:1428] Setting key eb2d5ad91e1a0f3417fd17566764adb5
Aug 06 09:29:08 dist-test-slave-45dx krb5kdc[29044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008548, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.2@KRBTEST.COM for kudu/127.20.113.62@KRBTEST.COM
I20260806 09:29:08.561471 29606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:08.543465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.2:57619 (local address 127.20.113.62:37355)
0806 09:29:08.544028 (+ 563us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:08.544036 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:08.544831 (+ 795us) server_negotiation.cc:407] Connection header received
0806 09:29:08.546677 (+ 1846us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:08.546682 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:08.546781 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:08.546915 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:08.548462 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:08.549383 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:08.551036 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:08.551633 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:08.551790 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:08.555637 (+ 3847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:29:08.555641 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:29:08.555645 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:29:08.555694 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:29:08.558268 (+ 2574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:08.559021 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:08.559026 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:08.559028 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:08.559158 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:08.559648 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:08.559659 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:08.559661 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:08.559926 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:29:08.560086 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:08.560819 (+ 733us) server_negotiation.cc:299] Negotiation successful
0806 09:29:08.561076 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":442,"thread_start_us":312,"threads_started":1}
I20260806 09:29:08.562476 29607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:08.543828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:37355 (local address 127.20.113.2:57619)
0806 09:29:08.544624 (+ 796us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:08.544672 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:08.546398 (+ 1726us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:08.547155 (+ 757us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:08.547163 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:08.547663 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:29:08.548234 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:08.548262 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:08.549540 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:08.549544 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:08.550870 (+ 1326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:08.550888 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:08.551887 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:08.551895 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:08.552133 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:08.552934 (+ 801us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:29:08.552957 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:29:08.555387 (+ 2430us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:29:08.558409 (+ 3022us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:29:08.558418 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:29:08.558434 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:29:08.558877 (+ 443us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:29:08.559276 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:29:08.559283 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:29:08.559286 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:29:08.559509 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:29:08.560060 (+ 551us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:29:08.560068 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:29:08.560458 (+ 390us) client_negotiation.cc:769] Sending connection context
0806 09:29:08.560719 (+ 261us) client_negotiation.cc:240] Negotiation successful
0806 09:29:08.561278 (+ 559us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":573,"thread_start_us":299,"threads_started":1}
I20260806 09:29:08.564075 29604 heartbeater.cc:347] Connected to a master server at 127.20.113.62:37355
I20260806 09:29:08.564451 29604 heartbeater.cc:464] Registering TS with master...
I20260806 09:29:08.565306 29604 heartbeater.cc:511] Master 127.20.113.62:37355 requested a full tablet report, sending...
I20260806 09:29:08.567081 29115 ts_manager.cc:194] Registered new tserver with Master: b5c5746b8fea498986dd89ad7759e31b (127.20.113.2:42921)
I20260806 09:29:08.568413 29115 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.113.2@KRBTEST.COM'} at 127.20.113.2:57619
WARNING: no policy specified for kudu/127.20.113.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.113.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.113.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.3.keytab.
Entry for principal kudu/127.20.113.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.3.keytab.
I20260806 09:29:08.613143 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786008495763054-20932-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786008495763054-20932-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786008495763054-20932-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786008495763054-20932-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.113.3:0
--local_ip_for_outbound_sockets=127.20.113.3
--webserver_interface=127.20.113.3
--webserver_port=0
--tserver_master_addrs=127.20.113.62:37355
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.3.keytab
--principal=kudu/127.20.113.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:35743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:29:08.960853 29611 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:29:08.961184 29611 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:29:08.961354 29611 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:29:08.968595 29611 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:29:08.968737 29611 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:29:08.968914 29611 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.113.3
I20260806 09:29:08.978544 29611 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:35743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786008495763054-20932-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786008495763054-20932-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.3.keytab
--principal=kudu/127.20.113.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.113.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786008495763054-20932-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.113.3
--webserver_port=0
--tserver_master_addrs=127.20.113.62:37355
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.113.3
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786008495763054-20932-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:29:08.979888 29611 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:29:08.981272 29611 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:29:08.994489 29618 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:29:08.994532 29621 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:29:08.994560 29619 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:29:08.995568 29611 server_base.cc:1034] running on GCE node
I20260806 09:29:08.996249 29611 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:29:08.997315 29611 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:29:08.998574 29611 hybrid_clock.cc:648] HybridClock initialized: now 1786008548998518 us; error 42 us; skew 500 ppm
I20260806 09:29:08.998965 29611 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:29:08 dist-test-slave-45dx krb5kdc[29044](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008548, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:29:09.002950 29611 init.cc:377] Logged in from keytab as kudu/127.20.113.3@KRBTEST.COM (short username kudu)
I20260806 09:29:09.004547 29611 webserver.cc:460] Webserver started at http://127.20.113.3:41133/ using document root <none> and password file <none>
I20260806 09:29:09.005231 29611 fs_manager.cc:359] Metadata directory not provided
I20260806 09:29:09.005371 29611 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:29:09.005703 29611 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:29:09.008656 29611 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786008495763054-20932-0/minicluster-data/ts-2/data/instance:
uuid: "706454a3967b4fbcb861ca370ef92625"
format_stamp: "Formatted at 2026-08-06 09:29:09 on dist-test-slave-45dx"
server_key: "5e17dd5b7cd3bd854714c4f4635d0912"
server_key_iv: "915fc702d931726ac037c904c20e27d8"
server_key_version: "encryptionkey@0"
I20260806 09:29:09.009526 29611 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786008495763054-20932-0/minicluster-data/ts-2/wal/instance:
uuid: "706454a3967b4fbcb861ca370ef92625"
format_stamp: "Formatted at 2026-08-06 09:29:09 on dist-test-slave-45dx"
server_key: "5e17dd5b7cd3bd854714c4f4635d0912"
server_key_iv: "915fc702d931726ac037c904c20e27d8"
server_key_version: "encryptionkey@0"
I20260806 09:29:09.015775 29611 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260806 09:29:09.019426 29635 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:29:09.020852 29611 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 09:29:09.021031 29611 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786008495763054-20932-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786008495763054-20932-0/minicluster-data/ts-2/wal
uuid: "706454a3967b4fbcb861ca370ef92625"
format_stamp: "Formatted at 2026-08-06 09:29:09 on dist-test-slave-45dx"
server_key: "5e17dd5b7cd3bd854714c4f4635d0912"
server_key_iv: "915fc702d931726ac037c904c20e27d8"
server_key_version: "encryptionkey@0"
I20260806 09:29:09.021236 29611 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786008495763054-20932-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786008495763054-20932-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786008495763054-20932-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:29:09.030923 29405 heartbeater.cc:503] Master 127.20.113.62:37355 was elected leader, sending a full tablet report...
I20260806 09:29:09.052362 29611 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:29:09.056993 29611 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:29:09.057297 29611 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:29:09.058286 29611 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:29:09.059686 29611 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:29:09.059763 29611 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:29:09.059940 29611 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:29:09.060009 29611 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:29:09.094710 29611 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.3:35749
I20260806 09:29:09.094817 29803 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.3:35749 every 8 connection(s)
I20260806 09:29:09.095999 29611 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786008495763054-20932-0/minicluster-data/ts-2/data/info.pb
I20260806 09:29:09.101297 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 29611
I20260806 09:29:09.101480 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786008495763054-20932-0/minicluster-data/ts-2/wal/instance
I20260806 09:29:09.101974 20932 external_mini_cluster.cc:1428] Setting key 743df77156f997af6d3eeede49772338
Aug 06 09:29:09 dist-test-slave-45dx krb5kdc[29044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008548, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.3@KRBTEST.COM for kudu/127.20.113.62@KRBTEST.COM
I20260806 09:29:09.115175 29807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:09.098537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.3:60053 (local address 127.20.113.62:37355)
0806 09:29:09.099007 (+ 470us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:09.099011 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:09.099806 (+ 795us) server_negotiation.cc:407] Connection header received
0806 09:29:09.101764 (+ 1958us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:09.101770 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:09.101872 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:09.102000 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:09.103694 (+ 1694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:09.104748 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:09.106417 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:09.106786 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:09.106886 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:09.110103 (+ 3217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:29:09.110108 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:29:09.110111 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:29:09.110140 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:29:09.112644 (+ 2504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:09.113313 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:09.113316 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:09.113319 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:09.113437 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:09.113813 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:09.113816 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:09.113818 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:09.114076 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:29:09.114189 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:09.114743 (+ 554us) server_negotiation.cc:299] Negotiation successful
0806 09:29:09.114901 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":240,"threads_started":1}
I20260806 09:29:09.116242 29808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:09.098952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:37355 (local address 127.20.113.3:60053)
0806 09:29:09.099648 (+ 696us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:09.099684 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:09.101426 (+ 1742us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:09.102238 (+ 812us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:09.102247 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:09.102840 (+ 593us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:29:09.103530 (+ 690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:09.103550 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:09.105039 (+ 1489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:09.105045 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:09.106286 (+ 1241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:09.106299 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:09.106978 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:09.106983 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:09.107171 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:09.107772 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:29:09.107793 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:29:09.109887 (+ 2094us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:29:09.112767 (+ 2880us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:29:09.112774 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:29:09.112803 (+ 29us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:29:09.113214 (+ 411us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:29:09.113535 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:29:09.113538 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:29:09.113540 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:29:09.113720 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:29:09.114180 (+ 460us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:29:09.114185 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:29:09.114457 (+ 272us) client_negotiation.cc:769] Sending connection context
0806 09:29:09.114674 (+ 217us) client_negotiation.cc:240] Negotiation successful
0806 09:29:09.115032 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":297,"threads_started":1}
I20260806 09:29:09.117758 29805 heartbeater.cc:347] Connected to a master server at 127.20.113.62:37355
I20260806 09:29:09.118031 29805 heartbeater.cc:464] Registering TS with master...
I20260806 09:29:09.118785 29805 heartbeater.cc:511] Master 127.20.113.62:37355 requested a full tablet report, sending...
I20260806 09:29:09.120151 29113 ts_manager.cc:194] Registered new tserver with Master: 706454a3967b4fbcb861ca370ef92625 (127.20.113.3:35749)
I20260806 09:29:09.121063 29113 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.113.3@KRBTEST.COM'} at 127.20.113.3:60053
I20260806 09:29:09.129434 20932 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786008495763054-20932-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786008495763054-20932-0/krb5kdc/test-admin.keytab.
Aug 06 09:29:09 dist-test-slave-45dx krb5kdc[29044](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008549, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:29:09.163802 20932 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260806 09:29:09.169314 29815 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 09:29:09.166577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49864 (local address 127.20.113.1:36637)
0806 09:29:09.167280 (+ 703us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:09.167287 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:09.167311 (+ 24us) server_negotiation.cc:407] Connection header received
0806 09:29:09.167378 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:09.167384 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:09.167603 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:09.167788 (+ 185us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:09.168794 (+ 1006us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49864: BlockingRecv error: recv got EOF from 127.0.0.1:49864 (error 108)
Metrics: {"server-negotiator.queue_time_us":562,"thread_start_us":356,"threads_started":1}
W20260806 09:29:09.169675 20932 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786008495763054-20932-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786008495763054-20932-0/krb5kdc/krb5cc)
Aug 06 09:29:09 dist-test-slave-45dx krb5kdc[29044](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008549, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:29:09.171304 20932 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 09:29:09 dist-test-slave-45dx krb5kdc[29044](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008549, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:29:09.172724 20932 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 09:29:09 dist-test-slave-45dx krb5kdc[29044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008549, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.113.1@KRBTEST.COM
I20260806 09:29:09.188692 29815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:09.174424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49876 (local address 127.20.113.1:36637)
0806 09:29:09.174628 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:09.174634 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:09.174777 (+ 143us) server_negotiation.cc:407] Connection header received
0806 09:29:09.174970 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:09.174980 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:09.175038 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:09.175137 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:09.176103 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:09.177207 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:09.178445 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:09.178999 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:09.179129 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:09.182776 (+ 3647us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:29:09.182788 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:29:09.182803 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:29:09.182842 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:29:09.185646 (+ 2804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:09.186381 (+ 735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:09.186391 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:09.186394 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:09.186527 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:09.186919 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:09.186929 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:09.186931 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:09.187292 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:29:09.187636 (+ 344us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:09.187994 (+ 358us) server_negotiation.cc:299] Negotiation successful
0806 09:29:09.188356 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
W20260806 09:29:09.194794 29815 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 09:29:09.193016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49882 (local address 127.20.113.1:36637)
0806 09:29:09.193246 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:09.193252 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:09.193490 (+ 238us) server_negotiation.cc:407] Connection header received
0806 09:29:09.193654 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:09.193657 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:09.193716 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:09.193795 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:09.194489 (+ 694us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49882: BlockingRecv error: recv got EOF from 127.0.0.1:49882 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
W20260806 09:29:09.194967 20932 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786008495763054-20932-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786008495763054-20932-0/krb5kdc/krb5cc)
Aug 06 09:29:09 dist-test-slave-45dx krb5kdc[29044](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008549, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:29:09.196563 20932 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 09:29:09 dist-test-slave-45dx krb5kdc[29044](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008549, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:29:09.197882 20932 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 09:29:09 dist-test-slave-45dx krb5kdc[29044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008549, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.113.1@KRBTEST.COM
I20260806 09:29:09.211395 29815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:09.199291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49892 (local address 127.20.113.1:36637)
0806 09:29:09.199488 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:09.199492 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:09.199567 (+ 75us) server_negotiation.cc:407] Connection header received
0806 09:29:09.199674 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:09.199677 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:09.199728 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:09.199801 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:09.200769 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:09.201473 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:09.202636 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:09.203077 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:09.203170 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:09.206655 (+ 3485us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:29:09.206672 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:29:09.206675 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:29:09.206718 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:29:09.209181 (+ 2463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:09.209766 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:09.209770 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:09.209772 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:09.209895 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:09.210206 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:09.210209 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:09.210211 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:09.210457 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:29:09.210675 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:09.210897 (+ 222us) server_negotiation.cc:299] Negotiation successful
0806 09:29:09.211103 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
W20260806 09:29:09.215764 29815 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 09:29:09.214027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49894 (local address 127.20.113.1:36637)
0806 09:29:09.214164 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:09.214168 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:09.214584 (+ 416us) server_negotiation.cc:407] Connection header received
0806 09:29:09.214681 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:09.214684 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:09.214749 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:09.214819 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:09.215514 (+ 695us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49894: BlockingRecv error: recv got EOF from 127.0.0.1:49894 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260806 09:29:09.215857 20932 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786008495763054-20932-0/krb5kdc/krb5cc' of type FILE
Aug 06 09:29:09 dist-test-slave-45dx krb5kdc[29044](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008549, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:29:09.217411 20932 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 09:29:09 dist-test-slave-45dx krb5kdc[29044](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008549, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:29:09.218664 20932 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 09:29:09 dist-test-slave-45dx krb5kdc[29044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008549, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.113.1@KRBTEST.COM
I20260806 09:29:09.232234 29815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:09.219911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49900 (local address 127.20.113.1:36637)
0806 09:29:09.220115 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:09.220120 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:09.220435 (+ 315us) server_negotiation.cc:407] Connection header received
0806 09:29:09.220614 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:09.220619 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:09.220679 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:09.220773 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:09.221635 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:09.222370 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:09.223610 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:09.224097 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:09.224222 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:09.227723 (+ 3501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:29:09.227727 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:29:09.227730 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:29:09.227759 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:29:09.230186 (+ 2427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:09.230725 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:09.230729 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:09.230731 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:09.230880 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:09.231191 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:09.231194 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:09.231196 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:09.231428 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:29:09.231650 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:09.231834 (+ 184us) server_negotiation.cc:299] Negotiation successful
0806 09:29:09.231993 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 09:29:09.233759 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 29210
I20260806 09:29:09.245592 29815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:09.234502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49912 (local address 127.20.113.1:36637)
0806 09:29:09.234679 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:09.234687 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:09.234978 (+ 291us) server_negotiation.cc:407] Connection header received
0806 09:29:09.235150 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:09.235161 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:09.235211 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:09.235306 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:09.236211 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:09.236954 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:09.238160 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:09.238574 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:09.238703 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:09.240673 (+ 1970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:29:09.240678 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:29:09.240681 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:29:09.240709 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:29:09.242945 (+ 2236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:09.243536 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:09.243541 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:09.243544 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:09.243674 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:09.244055 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:09.244060 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:09.244065 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:09.244404 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:29:09.244669 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:09.244965 (+ 296us) server_negotiation.cc:299] Negotiation successful
0806 09:29:09.245153 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 09:29:09.246281 29398 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:29:09.346271 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 29210
I20260806 09:29:09.374640 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 29411
Aug 06 09:29:09 dist-test-slave-45dx krb5kdc[29044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008549, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.113.2@KRBTEST.COM
I20260806 09:29:09.389982 29844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:09.375713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36684 (local address 127.20.113.2:42921)
0806 09:29:09.376440 (+ 727us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:09.376447 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:09.376474 (+ 27us) server_negotiation.cc:407] Connection header received
0806 09:29:09.376536 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:09.376542 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:09.376774 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:09.376905 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:09.377866 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:09.378814 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:09.379964 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:09.380556 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:09.380655 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:09.384428 (+ 3773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:29:09.384440 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:29:09.384454 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:29:09.384493 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:29:09.387237 (+ 2744us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:09.387894 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:09.387900 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:09.387903 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:09.388045 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:09.388427 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:09.388430 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:09.388432 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:09.388792 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:29:09.389038 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:09.389378 (+ 340us) server_negotiation.cc:299] Negotiation successful
0806 09:29:09.389650 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":505,"thread_start_us":377,"threads_started":1}
I20260806 09:29:09.391245 29597 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:29:09.473423 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 29411
I20260806 09:29:09.497027 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 29611
Aug 06 09:29:09 dist-test-slave-45dx krb5kdc[29044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008549, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.113.3@KRBTEST.COM
I20260806 09:29:09.512768 29846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:09.498274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56396 (local address 127.20.113.3:35749)
0806 09:29:09.498966 (+ 692us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:09.498973 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:09.499002 (+ 29us) server_negotiation.cc:407] Connection header received
0806 09:29:09.499062 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:09.499068 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:09.499273 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:09.499403 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:09.500472 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:09.501375 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:09.502609 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:09.503150 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:09.503244 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:09.507301 (+ 4057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:29:09.507312 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:29:09.507325 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:29:09.507356 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:29:09.510200 (+ 2844us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:09.510909 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:09.510916 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:09.510918 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:09.511037 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:09.511363 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:09.511366 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:09.511368 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:09.511692 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:29:09.511941 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:09.512134 (+ 193us) server_negotiation.cc:299] Negotiation successful
0806 09:29:09.512482 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":540,"thread_start_us":376,"threads_started":1}
I20260806 09:29:09.514150 29798 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:29:09.596575 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 29611
I20260806 09:29:09.621229 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 29062
I20260806 09:29:09.622562 29159 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:29:09.729635 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 29062
2026-08-06T09:29:09Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (4392 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 09:29:09 dist-test-slave-45dx krb5kdc[29852](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 09:29:09 dist-test-slave-45dx krb5kdc[29852](info): set up 2 sockets
Aug 06 09:29:09 dist-test-slave-45dx krb5kdc[29852](info): commencing operation
krb5kdc: starting...
W20260806 09:29:11.036898 20932 mini_kdc.cc:121] Time spent starting KDC: real 1.255s user 0.008s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/krb5kdc/test-user.keytab.
Aug 06 09:29:11 dist-test-slave-45dx krb5kdc[29852](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008551, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T09:29:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T09:29:11Z Disabled control of system clock
WARNING: no policy specified for oryx/127.20.113.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.113.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.113.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/krb5kdc/oryx_127.20.113.62.keytab.
Entry for principal oryx/127.20.113.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/krb5kdc/oryx_127.20.113.62.keytab.
I20260806 09:29:11.306062 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.113.62:45173
--webserver_interface=127.20.113.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/krb5kdc/oryx_127.20.113.62.keytab
--principal=oryx/127.20.113.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:41701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.113.62:45173
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:29:11.644814 29869 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:29:11.645146 29869 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:29:11.645298 29869 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:29:11.652494 29869 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 09:29:11.652643 29869 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:29:11.652730 29869 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:29:11.652822 29869 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 09:29:11.652905 29869 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 09:29:11.663053 29869 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:41701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.113.62:45173
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/krb5kdc/oryx_127.20.113.62.keytab
--principal=oryx/127.20.113.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.113.62:45173
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.113.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:29:11.664604 29869 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:29:11.665978 29869 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:29:11.677917 29880 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:29:11.677887 29875 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:29:11.678000 29877 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:29:11.679277 29869 server_base.cc:1034] running on GCE node
I20260806 09:29:11.680200 29869 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:29:11.681910 29869 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:29:11.683161 29869 hybrid_clock.cc:648] HybridClock initialized: now 1786008551683108 us; error 32 us; skew 500 ppm
I20260806 09:29:11.683579 29869 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:29:11 dist-test-slave-45dx krb5kdc[29852](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008551, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.113.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:29:11.686940 29869 init.cc:377] Logged in from keytab as oryx/127.20.113.62@KRBTEST.COM (short username oryx)
I20260806 09:29:11.688589 29869 webserver.cc:460] Webserver started at http://127.20.113.62:35413/ using document root <none> and password file <none>
I20260806 09:29:11.689352 29869 fs_manager.cc:359] Metadata directory not provided
I20260806 09:29:11.689515 29869 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:29:11.689841 29869 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:29:11.693130 29869 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/master-0/data/instance:
uuid: "0cbb5d09bd23462caa29c64dd33fdd25"
format_stamp: "Formatted at 2026-08-06 09:29:11 on dist-test-slave-45dx"
server_key: "c9befee3a1db6a866de2039f21bdd76b"
server_key_iv: "1d38e057a4a7ec413d68fe893d1d3d61"
server_key_version: "encryptionkey@0"
I20260806 09:29:11.693926 29869 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/master-0/wal/instance:
uuid: "0cbb5d09bd23462caa29c64dd33fdd25"
format_stamp: "Formatted at 2026-08-06 09:29:11 on dist-test-slave-45dx"
server_key: "c9befee3a1db6a866de2039f21bdd76b"
server_key_iv: "1d38e057a4a7ec413d68fe893d1d3d61"
server_key_version: "encryptionkey@0"
I20260806 09:29:11.699276 29869 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260806 09:29:11.703023 29889 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:29:11.704535 29869 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 09:29:11.704717 29869 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/master-0/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/master-0/wal
uuid: "0cbb5d09bd23462caa29c64dd33fdd25"
format_stamp: "Formatted at 2026-08-06 09:29:11 on dist-test-slave-45dx"
server_key: "c9befee3a1db6a866de2039f21bdd76b"
server_key_iv: "1d38e057a4a7ec413d68fe893d1d3d61"
server_key_version: "encryptionkey@0"
I20260806 09:29:11.704926 29869 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:29:11.719482 29869 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:29:11.723570 29869 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:29:11.723875 29869 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:29:11.742663 29869 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.62:45173
I20260806 09:29:11.742694 29969 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.62:45173 every 8 connection(s)
I20260806 09:29:11.744058 29869 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
I20260806 09:29:11.747674 29972 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:29:11.754371 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 29869
I20260806 09:29:11.754617 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/master-0/wal/instance
I20260806 09:29:11.755296 20932 external_mini_cluster.cc:1428] Setting key e394d4c98bf140ac47c829b50b97fd41
I20260806 09:29:11.757952 29972 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0cbb5d09bd23462caa29c64dd33fdd25: Bootstrap starting.
I20260806 09:29:11.761387 29972 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0cbb5d09bd23462caa29c64dd33fdd25: Neither blocks nor log segments found. Creating new log.
I20260806 09:29:11.762787 29972 log.cc:824] T 00000000000000000000000000000000 P 0cbb5d09bd23462caa29c64dd33fdd25: Log is configured to *not* fsync() on all Append() calls
I20260806 09:29:11.766037 29972 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0cbb5d09bd23462caa29c64dd33fdd25: No bootstrap required, opened a new log
Aug 06 09:29:11 dist-test-slave-45dx krb5kdc[29852](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008551, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM
I20260806 09:29:11.776746 29972 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0cbb5d09bd23462caa29c64dd33fdd25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cbb5d09bd23462caa29c64dd33fdd25" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 45173 } }
I20260806 09:29:11.777262 29972 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0cbb5d09bd23462caa29c64dd33fdd25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:29:11.777372 29972 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0cbb5d09bd23462caa29c64dd33fdd25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cbb5d09bd23462caa29c64dd33fdd25, State: Initialized, Role: FOLLOWER
I20260806 09:29:11.778054 29972 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0cbb5d09bd23462caa29c64dd33fdd25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cbb5d09bd23462caa29c64dd33fdd25" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 45173 } }
I20260806 09:29:11.778381 29972 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0cbb5d09bd23462caa29c64dd33fdd25 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 09:29:11.778542 29972 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0cbb5d09bd23462caa29c64dd33fdd25 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 09:29:11.778733 29972 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0cbb5d09bd23462caa29c64dd33fdd25 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:29:11.780238 29976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:11.757251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39442 (local address 127.20.113.62:45173)
0806 09:29:11.758098 (+ 847us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:11.758116 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:11.758168 (+ 52us) server_negotiation.cc:407] Connection header received
0806 09:29:11.759902 (+ 1734us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:11.759926 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:11.760264 (+ 338us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:11.760621 (+ 357us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:11.762580 (+ 1959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:11.763975 (+ 1395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:11.765735 (+ 1760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:11.766489 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:11.766632 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:11.771230 (+ 4598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:29:11.771241 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:29:11.771259 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:29:11.771323 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:29:11.775446 (+ 4123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:11.776201 (+ 755us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:11.776209 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:11.776217 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:11.776455 (+ 238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:11.776851 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:11.776858 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:11.776861 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:11.777602 (+ 741us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:29:11.777919 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:11.778394 (+ 475us) server_negotiation.cc:299] Negotiation successful
0806 09:29:11.778856 (+ 462us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":598,"thread_start_us":405,"threads_started":1}
I20260806 09:29:11.781628 29972 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0cbb5d09bd23462caa29c64dd33fdd25 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cbb5d09bd23462caa29c64dd33fdd25" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 45173 } }
I20260806 09:29:11.782132 29972 leader_election.cc:302] T 00000000000000000000000000000000 P 0cbb5d09bd23462caa29c64dd33fdd25 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0cbb5d09bd23462caa29c64dd33fdd25; no voters:
I20260806 09:29:11.782840 29972 leader_election.cc:288] T 00000000000000000000000000000000 P 0cbb5d09bd23462caa29c64dd33fdd25 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 09:29:11.783026 29978 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0cbb5d09bd23462caa29c64dd33fdd25 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:29:11.783926 29978 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0cbb5d09bd23462caa29c64dd33fdd25 [term 1 LEADER]: Becoming Leader. State: Replica: 0cbb5d09bd23462caa29c64dd33fdd25, State: Running, Role: LEADER
I20260806 09:29:11.784559 29978 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0cbb5d09bd23462caa29c64dd33fdd25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cbb5d09bd23462caa29c64dd33fdd25" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 45173 } }
I20260806 09:29:11.785305 29972 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0cbb5d09bd23462caa29c64dd33fdd25 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 09:29:11.788329 29979 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0cbb5d09bd23462caa29c64dd33fdd25 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0cbb5d09bd23462caa29c64dd33fdd25" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cbb5d09bd23462caa29c64dd33fdd25" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 45173 } } }
I20260806 09:29:11.788282 29980 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0cbb5d09bd23462caa29c64dd33fdd25 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0cbb5d09bd23462caa29c64dd33fdd25. Latest consensus state: current_term: 1 leader_uuid: "0cbb5d09bd23462caa29c64dd33fdd25" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cbb5d09bd23462caa29c64dd33fdd25" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 45173 } } }
I20260806 09:29:11.788728 29979 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0cbb5d09bd23462caa29c64dd33fdd25 [sys.catalog]: This master's current role is: LEADER
I20260806 09:29:11.788807 29980 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0cbb5d09bd23462caa29c64dd33fdd25 [sys.catalog]: This master's current role is: LEADER
I20260806 09:29:11.796886 29990 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 09:29:11.802220 29990 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 09:29:11.811141 29990 catalog_manager.cc:1347] Generated new cluster ID: e50f8b022b59446eaf1c3aaefb8eb3e7
I20260806 09:29:11.811271 29990 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 09:29:11.830399 29990 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 09:29:11.831714 29990 catalog_manager.cc:1504] Loading token signing keys...
I20260806 09:29:11.839651 29990 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0cbb5d09bd23462caa29c64dd33fdd25: Generated new TSK 0
I20260806 09:29:11.840495 29990 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.20.113.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.113.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.113.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/krb5kdc/oryx_127.20.113.1.keytab.
Entry for principal oryx/127.20.113.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/krb5kdc/oryx_127.20.113.1.keytab.
I20260806 09:29:11.915365 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.113.1:0
--local_ip_for_outbound_sockets=127.20.113.1
--webserver_interface=127.20.113.1
--webserver_port=0
--tserver_master_addrs=127.20.113.62:45173
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/krb5kdc/oryx_127.20.113.1.keytab
--principal=oryx/127.20.113.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:41701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:29:12.258234 30010 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:29:12.258567 30010 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:29:12.258742 30010 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:29:12.270056 30010 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:29:12.270202 30010 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:29:12.270378 30010 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.113.1
I20260806 09:29:12.281072 30010 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:41701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/krb5kdc/oryx_127.20.113.1.keytab
--principal=oryx/127.20.113.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.113.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.113.1
--webserver_port=0
--tserver_master_addrs=127.20.113.62:45173
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.113.1
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:29:12.282455 30010 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:29:12.283736 30010 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:29:12.297199 30019 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:29:12.297250 30024 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:29:12.297273 30028 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:29:12.298317 30010 server_base.cc:1034] running on GCE node
I20260806 09:29:12.298957 30010 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:29:12.299947 30010 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:29:12.301182 30010 hybrid_clock.cc:648] HybridClock initialized: now 1786008552301148 us; error 74 us; skew 500 ppm
I20260806 09:29:12.301502 30010 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:29:12 dist-test-slave-45dx krb5kdc[29852](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008552, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.113.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:29:12.304972 30010 init.cc:377] Logged in from keytab as oryx/127.20.113.1@KRBTEST.COM (short username oryx)
I20260806 09:29:12.306351 30010 webserver.cc:460] Webserver started at http://127.20.113.1:41597/ using document root <none> and password file <none>
I20260806 09:29:12.307040 30010 fs_manager.cc:359] Metadata directory not provided
I20260806 09:29:12.307199 30010 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:29:12.307508 30010 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:29:12.310451 30010 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/ts-0/data/instance:
uuid: "f151cc51113947f6a0ea2f463ed20681"
format_stamp: "Formatted at 2026-08-06 09:29:12 on dist-test-slave-45dx"
server_key: "cd9e3d7a793e12dff6531e4b8724e1d1"
server_key_iv: "3ffa1a02aabe475403069673efde3baf"
server_key_version: "encryptionkey@0"
I20260806 09:29:12.311213 30010 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/ts-0/wal/instance:
uuid: "f151cc51113947f6a0ea2f463ed20681"
format_stamp: "Formatted at 2026-08-06 09:29:12 on dist-test-slave-45dx"
server_key: "cd9e3d7a793e12dff6531e4b8724e1d1"
server_key_iv: "3ffa1a02aabe475403069673efde3baf"
server_key_version: "encryptionkey@0"
I20260806 09:29:12.316151 30010 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.006s
I20260806 09:29:12.319440 30037 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:29:12.320779 30010 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 09:29:12.320955 30010 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/ts-0/wal
uuid: "f151cc51113947f6a0ea2f463ed20681"
format_stamp: "Formatted at 2026-08-06 09:29:12 on dist-test-slave-45dx"
server_key: "cd9e3d7a793e12dff6531e4b8724e1d1"
server_key_iv: "3ffa1a02aabe475403069673efde3baf"
server_key_version: "encryptionkey@0"
I20260806 09:29:12.321151 30010 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:29:12.339393 30010 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:29:12.343811 30010 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:29:12.344128 30010 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:29:12.345144 30010 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:29:12.346489 30010 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:29:12.346593 30010 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:29:12.346740 30010 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:29:12.346812 30010 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:29:12.380240 30010 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.1:37689
I20260806 09:29:12.380282 30190 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.1:37689 every 8 connection(s)
I20260806 09:29:12.381577 30010 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/ts-0/data/info.pb
I20260806 09:29:12.392120 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 30010
I20260806 09:29:12.392410 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/ts-0/wal/instance
I20260806 09:29:12.393157 20932 external_mini_cluster.cc:1428] Setting key e7b41750531438f5dc793461ad0ecbfb
Aug 06 09:29:12 dist-test-slave-45dx krb5kdc[29852](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008552, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.113.1@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM
I20260806 09:29:12.402377 30196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:12.384130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.1:44283 (local address 127.20.113.62:45173)
0806 09:29:12.384667 (+ 537us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:12.384671 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:12.385495 (+ 824us) server_negotiation.cc:407] Connection header received
0806 09:29:12.387698 (+ 2203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:12.387703 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:12.387803 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:12.387930 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:12.389466 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:12.390177 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:12.391737 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:12.392134 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:12.392287 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:12.396415 (+ 4128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:29:12.396420 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:29:12.396423 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:29:12.396455 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:29:12.399140 (+ 2685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:12.399894 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:12.399906 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:12.399908 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:12.400028 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:12.400613 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:12.400617 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:12.400619 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:12.400867 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:29:12.401023 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:12.401734 (+ 711us) server_negotiation.cc:299] Negotiation successful
0806 09:29:12.402001 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":425,"thread_start_us":313,"threads_started":1}
I20260806 09:29:12.403367 30197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:12.384600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:45173 (local address 127.20.113.1:44283)
0806 09:29:12.385319 (+ 719us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:12.385357 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:12.387382 (+ 2025us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:12.388167 (+ 785us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:12.388175 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:12.388740 (+ 565us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:29:12.389285 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:12.389302 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:12.390365 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:12.390369 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:12.391540 (+ 1171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:12.391551 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:12.392397 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:12.392407 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:12.392697 (+ 290us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:12.393455 (+ 758us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:29:12.393483 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:29:12.396171 (+ 2688us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:29:12.399304 (+ 3133us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:29:12.399313 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:29:12.399329 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:29:12.399756 (+ 427us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:29:12.400164 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:29:12.400169 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:29:12.400173 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:29:12.400472 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:29:12.401000 (+ 528us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:29:12.401008 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:29:12.401352 (+ 344us) client_negotiation.cc:769] Sending connection context
0806 09:29:12.401620 (+ 268us) client_negotiation.cc:240] Negotiation successful
0806 09:29:12.402192 (+ 572us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":500,"thread_start_us":283,"threads_started":1}
I20260806 09:29:12.405015 30192 heartbeater.cc:347] Connected to a master server at 127.20.113.62:45173
I20260806 09:29:12.405359 30192 heartbeater.cc:464] Registering TS with master...
I20260806 09:29:12.406172 30192 heartbeater.cc:511] Master 127.20.113.62:45173 requested a full tablet report, sending...
I20260806 09:29:12.408447 29915 ts_manager.cc:194] Registered new tserver with Master: f151cc51113947f6a0ea2f463ed20681 (127.20.113.1:37689)
I20260806 09:29:12.410034 29915 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.20.113.1@KRBTEST.COM'} at 127.20.113.1:44283
WARNING: no policy specified for oryx/127.20.113.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.113.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.113.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/krb5kdc/oryx_127.20.113.2.keytab.
Entry for principal oryx/127.20.113.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/krb5kdc/oryx_127.20.113.2.keytab.
I20260806 09:29:12.459089 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.113.2:0
--local_ip_for_outbound_sockets=127.20.113.2
--webserver_interface=127.20.113.2
--webserver_port=0
--tserver_master_addrs=127.20.113.62:45173
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/krb5kdc/oryx_127.20.113.2.keytab
--principal=oryx/127.20.113.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:41701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:29:12.798848 30200 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:29:12.799180 30200 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:29:12.799276 30200 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:29:12.806474 30200 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:29:12.806578 30200 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:29:12.806685 30200 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.113.2
I20260806 09:29:12.815980 30200 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:41701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/krb5kdc/oryx_127.20.113.2.keytab
--principal=oryx/127.20.113.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.113.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.113.2
--webserver_port=0
--tserver_master_addrs=127.20.113.62:45173
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.113.2
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:29:12.817337 30200 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:29:12.818626 30200 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:29:12.832409 30210 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:29:12.832392 30213 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:29:12.832463 30211 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:29:12.837785 30200 server_base.cc:1034] running on GCE node
I20260806 09:29:12.838421 30200 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:29:12.839393 30200 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:29:12.840600 30200 hybrid_clock.cc:648] HybridClock initialized: now 1786008552840545 us; error 40 us; skew 500 ppm
I20260806 09:29:12.840900 30200 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:29:12 dist-test-slave-45dx krb5kdc[29852](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008552, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.113.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:29:12.844107 30200 init.cc:377] Logged in from keytab as oryx/127.20.113.2@KRBTEST.COM (short username oryx)
I20260806 09:29:12.845665 30200 webserver.cc:460] Webserver started at http://127.20.113.2:46583/ using document root <none> and password file <none>
I20260806 09:29:12.846340 30200 fs_manager.cc:359] Metadata directory not provided
I20260806 09:29:12.846446 30200 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:29:12.846700 30200 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:29:12.849673 30200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/ts-1/data/instance:
uuid: "af2db7ab1bbc490f9bceae1317438337"
format_stamp: "Formatted at 2026-08-06 09:29:12 on dist-test-slave-45dx"
server_key: "7a04c34aded4b9ffb27c8afe515b7bc9"
server_key_iv: "fffd4978c7a08102e48407cf5479a707"
server_key_version: "encryptionkey@0"
I20260806 09:29:12.850428 30200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/ts-1/wal/instance:
uuid: "af2db7ab1bbc490f9bceae1317438337"
format_stamp: "Formatted at 2026-08-06 09:29:12 on dist-test-slave-45dx"
server_key: "7a04c34aded4b9ffb27c8afe515b7bc9"
server_key_iv: "fffd4978c7a08102e48407cf5479a707"
server_key_version: "encryptionkey@0"
I20260806 09:29:12.855700 30200 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260806 09:29:12.859294 30226 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:29:12.860694 30200 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 09:29:12.860857 30200 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/ts-1/wal
uuid: "af2db7ab1bbc490f9bceae1317438337"
format_stamp: "Formatted at 2026-08-06 09:29:12 on dist-test-slave-45dx"
server_key: "7a04c34aded4b9ffb27c8afe515b7bc9"
server_key_iv: "fffd4978c7a08102e48407cf5479a707"
server_key_version: "encryptionkey@0"
I20260806 09:29:12.861124 30200 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:29:12.878221 30200 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:29:12.882184 30200 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:29:12.882477 30200 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:29:12.883486 30200 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:29:12.884925 30200 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:29:12.884999 30200 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:29:12.885128 30200 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:29:12.885177 30200 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:29:12.920382 30200 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.2:38959
I20260806 09:29:12.920425 30388 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.2:38959 every 8 connection(s)
I20260806 09:29:12.921705 30200 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/ts-1/data/info.pb
I20260806 09:29:12.926311 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 30200
I20260806 09:29:12.926496 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/ts-1/wal/instance
I20260806 09:29:12.926956 20932 external_mini_cluster.cc:1428] Setting key 502ee960f4fe93d59856a0d47b7151e3
Aug 06 09:29:12 dist-test-slave-45dx krb5kdc[29852](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008552, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.113.2@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM
I20260806 09:29:12.942130 30392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:12.924291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.2:59115 (local address 127.20.113.62:45173)
0806 09:29:12.924834 (+ 543us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:12.924840 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:12.925829 (+ 989us) server_negotiation.cc:407] Connection header received
0806 09:29:12.927821 (+ 1992us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:12.927827 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:12.927921 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:12.928052 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:12.929611 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:12.930345 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:12.931954 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:12.932466 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:12.932600 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:12.936254 (+ 3654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:29:12.936260 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:29:12.936263 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:29:12.936294 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:29:12.939011 (+ 2717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:12.939749 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:12.939755 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:12.939759 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:12.939906 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:12.940348 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:12.940353 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:12.940356 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:12.940624 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:29:12.940781 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:12.941476 (+ 695us) server_negotiation.cc:299] Negotiation successful
0806 09:29:12.941700 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":420,"thread_start_us":236,"threads_started":1}
I20260806 09:29:12.943073 30394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:12.924975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:45173 (local address 127.20.113.2:59115)
0806 09:29:12.925644 (+ 669us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:12.925687 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:12.927555 (+ 1868us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:12.928297 (+ 742us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:12.928352 (+ 55us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:12.928879 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:29:12.929430 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:12.929447 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:12.930534 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:12.930538 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:12.931784 (+ 1246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:12.931796 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:12.932723 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:12.932730 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:12.932995 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:12.933699 (+ 704us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:29:12.933725 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:29:12.936013 (+ 2288us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:29:12.939160 (+ 3147us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:29:12.939170 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:29:12.939185 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:29:12.939621 (+ 436us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:29:12.940020 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:29:12.940024 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:29:12.940028 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:29:12.940214 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:29:12.940754 (+ 540us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:29:12.940761 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:29:12.941088 (+ 327us) client_negotiation.cc:769] Sending connection context
0806 09:29:12.941378 (+ 290us) client_negotiation.cc:240] Negotiation successful
0806 09:29:12.941949 (+ 571us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":246,"threads_started":1}
I20260806 09:29:12.944674 30389 heartbeater.cc:347] Connected to a master server at 127.20.113.62:45173
I20260806 09:29:12.945036 30389 heartbeater.cc:464] Registering TS with master...
I20260806 09:29:12.945976 30389 heartbeater.cc:511] Master 127.20.113.62:45173 requested a full tablet report, sending...
I20260806 09:29:12.947722 29915 ts_manager.cc:194] Registered new tserver with Master: af2db7ab1bbc490f9bceae1317438337 (127.20.113.2:38959)
I20260806 09:29:12.949074 29915 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.20.113.2@KRBTEST.COM'} at 127.20.113.2:59115
WARNING: no policy specified for oryx/127.20.113.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.113.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.113.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/krb5kdc/oryx_127.20.113.3.keytab.
Entry for principal oryx/127.20.113.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/krb5kdc/oryx_127.20.113.3.keytab.
I20260806 09:29:12.990873 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.113.3:0
--local_ip_for_outbound_sockets=127.20.113.3
--webserver_interface=127.20.113.3
--webserver_port=0
--tserver_master_addrs=127.20.113.62:45173
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/krb5kdc/oryx_127.20.113.3.keytab
--principal=oryx/127.20.113.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:41701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:29:13.338701 30397 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:29:13.339006 30397 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:29:13.339105 30397 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:29:13.346494 30397 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:29:13.346616 30397 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:29:13.346738 30397 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.113.3
I20260806 09:29:13.356508 30397 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:41701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/krb5kdc/oryx_127.20.113.3.keytab
--principal=oryx/127.20.113.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.113.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.113.3
--webserver_port=0
--tserver_master_addrs=127.20.113.62:45173
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.113.3
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:29:13.357757 30397 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:29:13.359225 30397 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:29:13.372627 30404 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:29:13.372671 30409 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:29:13.372701 30405 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:29:13.373497 30397 server_base.cc:1034] running on GCE node
I20260806 09:29:13.374085 30397 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:29:13.374995 30397 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:29:13.376200 30397 hybrid_clock.cc:648] HybridClock initialized: now 1786008553376135 us; error 58 us; skew 500 ppm
I20260806 09:29:13.376569 30397 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:29:13 dist-test-slave-45dx krb5kdc[29852](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008553, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.113.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:29:13.379779 30397 init.cc:377] Logged in from keytab as oryx/127.20.113.3@KRBTEST.COM (short username oryx)
I20260806 09:29:13.381513 30397 webserver.cc:460] Webserver started at http://127.20.113.3:42243/ using document root <none> and password file <none>
I20260806 09:29:13.382222 30397 fs_manager.cc:359] Metadata directory not provided
I20260806 09:29:13.382325 30397 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:29:13.382802 30397 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:29:13.386054 30397 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/ts-2/data/instance:
uuid: "fb643e201d404a53a320740b57e36912"
format_stamp: "Formatted at 2026-08-06 09:29:13 on dist-test-slave-45dx"
server_key: "21113f96590945ac003474635bd5f5d1"
server_key_iv: "abbf4777e776df873a0427d3297b5519"
server_key_version: "encryptionkey@0"
I20260806 09:29:13.386967 30397 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/ts-2/wal/instance:
uuid: "fb643e201d404a53a320740b57e36912"
format_stamp: "Formatted at 2026-08-06 09:29:13 on dist-test-slave-45dx"
server_key: "21113f96590945ac003474635bd5f5d1"
server_key_iv: "abbf4777e776df873a0427d3297b5519"
server_key_version: "encryptionkey@0"
I20260806 09:29:13.392017 30397 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260806 09:29:13.395483 30418 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:29:13.396920 30397 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260806 09:29:13.397091 30397 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/ts-2/wal
uuid: "fb643e201d404a53a320740b57e36912"
format_stamp: "Formatted at 2026-08-06 09:29:13 on dist-test-slave-45dx"
server_key: "21113f96590945ac003474635bd5f5d1"
server_key_iv: "abbf4777e776df873a0427d3297b5519"
server_key_version: "encryptionkey@0"
I20260806 09:29:13.397293 30397 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:29:13.413582 30192 heartbeater.cc:503] Master 127.20.113.62:45173 was elected leader, sending a full tablet report...
I20260806 09:29:13.414865 30397 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:29:13.419006 30397 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:29:13.419304 30397 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:29:13.420266 30397 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:29:13.421672 30397 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:29:13.421792 30397 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:29:13.421952 30397 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:29:13.422034 30397 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:29:13.457875 30397 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.3:35129
I20260806 09:29:13.457968 30589 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.3:35129 every 8 connection(s)
I20260806 09:29:13.459216 30397 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/ts-2/data/info.pb
I20260806 09:29:13.468407 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 30397
I20260806 09:29:13.468623 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/ts-2/wal/instance
I20260806 09:29:13.469130 20932 external_mini_cluster.cc:1428] Setting key 0b3b15bc73236f862a1e5e4971ffdffb
Aug 06 09:29:13 dist-test-slave-45dx krb5kdc[29852](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008553, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.113.3@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM
I20260806 09:29:13.480401 30593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:13.461911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.3:52553 (local address 127.20.113.62:45173)
0806 09:29:13.462431 (+ 520us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:13.462441 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:13.463292 (+ 851us) server_negotiation.cc:407] Connection header received
0806 09:29:13.465422 (+ 2130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:13.465427 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:13.465517 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:13.465620 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:13.467194 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:13.468127 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:13.469932 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:13.470480 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:13.470619 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:13.474309 (+ 3690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:29:13.474320 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:29:13.474324 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:29:13.474365 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:29:13.477303 (+ 2938us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:13.478027 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:13.478034 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:13.478037 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:13.478179 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:13.478622 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:13.478631 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:13.478633 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:13.478912 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:29:13.479076 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:13.479725 (+ 649us) server_negotiation.cc:299] Negotiation successful
0806 09:29:13.479951 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":332,"threads_started":1}
I20260806 09:29:13.481289 30596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:13.462355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:45173 (local address 127.20.113.3:52553)
0806 09:29:13.463115 (+ 760us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:13.463162 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:13.465108 (+ 1946us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:13.465882 (+ 774us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:13.465891 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:13.466431 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:29:13.467014 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:13.467032 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:13.468279 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:13.468283 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:13.469771 (+ 1488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:13.469784 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:13.470738 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:13.470746 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:13.470977 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:13.471692 (+ 715us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:29:13.471721 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:29:13.474100 (+ 2379us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:29:13.477454 (+ 3354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:29:13.477463 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:29:13.477479 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:29:13.477895 (+ 416us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:29:13.478301 (+ 406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:29:13.478306 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:29:13.478314 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:29:13.478501 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:29:13.479041 (+ 540us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:29:13.479049 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:29:13.479359 (+ 310us) client_negotiation.cc:769] Sending connection context
0806 09:29:13.479621 (+ 262us) client_negotiation.cc:240] Negotiation successful
0806 09:29:13.480153 (+ 532us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":511,"thread_start_us":280,"threads_started":1}
I20260806 09:29:13.482743 30591 heartbeater.cc:347] Connected to a master server at 127.20.113.62:45173
I20260806 09:29:13.483121 30591 heartbeater.cc:464] Registering TS with master...
I20260806 09:29:13.484084 30591 heartbeater.cc:511] Master 127.20.113.62:45173 requested a full tablet report, sending...
I20260806 09:29:13.485584 29915 ts_manager.cc:194] Registered new tserver with Master: fb643e201d404a53a320740b57e36912 (127.20.113.3:35129)
I20260806 09:29:13.486346 29915 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.20.113.3@KRBTEST.COM'} at 127.20.113.3:52553
I20260806 09:29:13.496493 20932 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.20.113.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.113.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.113.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/krb5kdc/oryx_127.20.113.61.keytab.
Entry for principal oryx/127.20.113.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/krb5kdc/oryx_127.20.113.61.keytab.
I20260806 09:29:13.555753 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.113.61:41373
--webserver_interface=127.20.113.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/krb5kdc/oryx_127.20.113.61.keytab
--principal=oryx/127.20.113.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:41701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.113.62:45173,127.20.113.61:41373
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:29:13.913802 30601 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:29:13.914081 30601 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:29:13.914278 30601 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:29:13.922008 30601 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 09:29:13.922205 30601 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:29:13.922319 30601 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:29:13.922447 30601 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 09:29:13.922529 30601 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 09:29:13.934052 30601 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:41701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.20.113.62:45173,127.20.113.61:41373
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/krb5kdc/oryx_127.20.113.61.keytab
--principal=oryx/127.20.113.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.113.61:41373
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.20.113.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
W20260806 09:29:13.934541 30601 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.20.113.62:45173,127.20.113.61:41373'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260806 09:29:13.935710 30601 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:29:13.937143 30601 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:29:13.949232 30610 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:29:13.949307 30608 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:29:13.949354 30612 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:29:13.952262 30389 heartbeater.cc:503] Master 127.20.113.62:45173 was elected leader, sending a full tablet report...
I20260806 09:29:13.953876 30601 server_base.cc:1034] running on GCE node
I20260806 09:29:13.954777 30601 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:29:13.956163 30601 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:29:13.957422 30601 hybrid_clock.cc:648] HybridClock initialized: now 1786008553957352 us; error 57 us; skew 500 ppm
I20260806 09:29:13.957759 30601 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:29:13 dist-test-slave-45dx krb5kdc[29852](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008553, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.113.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:29:13.961238 30601 init.cc:377] Logged in from keytab as oryx/127.20.113.61@KRBTEST.COM (short username oryx)
I20260806 09:29:13.962852 30601 webserver.cc:460] Webserver started at http://127.20.113.61:41295/ using document root <none> and password file <none>
I20260806 09:29:13.963640 30601 fs_manager.cc:359] Metadata directory not provided
I20260806 09:29:13.963730 30601 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:29:13.963979 30601 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:29:13.967080 30601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/master-1/data/instance:
uuid: "377f07a0afaa4800aa98d87714775150"
format_stamp: "Formatted at 2026-08-06 09:29:13 on dist-test-slave-45dx"
server_key: "3ba94b12271f82bfd9e3dfd83172fb08"
server_key_iv: "8fb0b78efbd3af269b272eba60edbb7a"
server_key_version: "encryptionkey@0"
I20260806 09:29:13.967923 30601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/master-1/wal/instance:
uuid: "377f07a0afaa4800aa98d87714775150"
format_stamp: "Formatted at 2026-08-06 09:29:13 on dist-test-slave-45dx"
server_key: "3ba94b12271f82bfd9e3dfd83172fb08"
server_key_iv: "8fb0b78efbd3af269b272eba60edbb7a"
server_key_version: "encryptionkey@0"
I20260806 09:29:13.973194 30601 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260806 09:29:13.976640 30624 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:29:13.977988 30601 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 09:29:13.978196 30601 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/master-1/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/master-1/wal
uuid: "377f07a0afaa4800aa98d87714775150"
format_stamp: "Formatted at 2026-08-06 09:29:13 on dist-test-slave-45dx"
server_key: "3ba94b12271f82bfd9e3dfd83172fb08"
server_key_iv: "8fb0b78efbd3af269b272eba60edbb7a"
server_key_version: "encryptionkey@0"
I20260806 09:29:13.978438 30601 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:29:13.990762 30601 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:29:13.995353 30601 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:29:13.995636 30601 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:29:14.017138 30601 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.61:41373
I20260806 09:29:14.017143 30703 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.61:41373 every 8 connection(s)
I20260806 09:29:14.018595 30601 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/master-1/data/info.pb
I20260806 09:29:14.021616 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 30601
I20260806 09:29:14.022362 30704 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:29:14.033560 30704 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.113.62" port: 45173 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 09:29:14 dist-test-slave-45dx krb5kdc[29852](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008551, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.20.113.61@KRBTEST.COM
Aug 06 09:29:14 dist-test-slave-45dx krb5kdc[29852](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008553, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.113.61@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM
I20260806 09:29:14.048585 30707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:14.023844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43010 (local address 127.20.113.61:41373)
0806 09:29:14.024775 (+ 931us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:14.024789 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:14.024837 (+ 48us) server_negotiation.cc:407] Connection header received
0806 09:29:14.027422 (+ 2585us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:14.027450 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:14.027995 (+ 545us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:14.028457 (+ 462us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:14.030318 (+ 1861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:14.032591 (+ 2273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:14.033918 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:14.034680 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:14.034866 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:14.039869 (+ 5003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:29:14.039876 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:29:14.039886 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:29:14.039923 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:29:14.043706 (+ 3783us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:14.044593 (+ 887us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:14.044604 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:14.044613 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:14.044814 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:14.045275 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:14.045279 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:14.045281 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:14.045820 (+ 539us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:29:14.046112 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:14.046603 (+ 491us) server_negotiation.cc:299] Negotiation successful
0806 09:29:14.047021 (+ 418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":718,"thread_start_us":492,"threads_started":1}
I20260806 09:29:14.052989 30713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:14.036143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:45173 (local address 127.0.0.1:59150)
0806 09:29:14.036972 (+ 829us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:14.037005 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:14.037192 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:14.037596 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:14.037603 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:14.038241 (+ 638us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:29:14.038596 (+ 355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:14.038610 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:14.039732 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:14.039738 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:14.041038 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:14.041051 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:14.042154 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:14.042159 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:14.042322 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:14.042960 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:29:14.042977 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:29:14.045919 (+ 2942us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:29:14.050055 (+ 4136us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:29:14.050064 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:29:14.050070 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:29:14.050766 (+ 696us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:29:14.051149 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:29:14.051153 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:29:14.051157 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:29:14.051336 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:29:14.052096 (+ 760us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:29:14.052101 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:29:14.052248 (+ 147us) client_negotiation.cc:769] Sending connection context
0806 09:29:14.052468 (+ 220us) client_negotiation.cc:240] Negotiation successful
0806 09:29:14.052700 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":640,"thread_start_us":489,"threads_started":1}
I20260806 09:29:14.052996 30708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:14.036062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59150 (local address 127.20.113.62:45173)
0806 09:29:14.036596 (+ 534us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:14.036605 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:14.037111 (+ 506us) server_negotiation.cc:407] Connection header received
0806 09:29:14.037351 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:14.037355 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:14.037425 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:14.037537 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:14.038765 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:14.039555 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:14.041196 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:14.041945 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:14.042086 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:14.046132 (+ 4046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:29:14.046140 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:29:14.046145 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:29:14.046189 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:29:14.049894 (+ 3705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:14.050899 (+ 1005us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:14.050903 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:14.050905 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:14.051035 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:14.051497 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:14.051505 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:14.051508 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:14.051976 (+ 468us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:29:14.052093 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:14.052575 (+ 482us) server_negotiation.cc:299] Negotiation successful
0806 09:29:14.052747 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":398,"thread_start_us":315,"threads_started":1}
I20260806 09:29:14.054738 30704 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.113.61" port: 41373 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 09:29:14 dist-test-slave-45dx krb5kdc[29852](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008553, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.113.61@KRBTEST.COM for oryx/127.20.113.61@KRBTEST.COM
I20260806 09:29:14.067016 30713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:14.055606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.61:41373 (local address 127.0.0.1:43022)
0806 09:29:14.055760 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:14.055774 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:14.055896 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:14.056175 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:14.056179 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:14.056585 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:29:14.056723 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:14.056732 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:14.057708 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:14.057713 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:14.058560 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:14.058570 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:14.059233 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:14.059239 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:14.059421 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:14.059925 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:29:14.059942 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:29:14.061912 (+ 1970us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:29:14.064936 (+ 3024us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:29:14.064940 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:29:14.064943 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:29:14.065294 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:29:14.065701 (+ 407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:29:14.065704 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:29:14.065706 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:29:14.065832 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:29:14.066333 (+ 501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:29:14.066337 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:29:14.066469 (+ 132us) client_negotiation.cc:769] Sending connection context
0806 09:29:14.066578 (+ 109us) client_negotiation.cc:240] Negotiation successful
0806 09:29:14.066780 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43,"mutex_wait_us":1}
I20260806 09:29:14.067252 30707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:14.055493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43022 (local address 127.20.113.61:41373)
0806 09:29:14.055679 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:14.055685 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:14.055836 (+ 151us) server_negotiation.cc:407] Connection header received
0806 09:29:14.056013 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:14.056017 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:14.056066 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:14.056159 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:14.056852 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:14.057567 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:14.058700 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:14.059087 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:14.059222 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:14.062064 (+ 2842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:29:14.062068 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:29:14.062071 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:29:14.062100 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:29:14.064816 (+ 2716us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:14.065464 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:14.065469 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:14.065477 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:14.065603 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:14.065942 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:14.065945 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:14.065947 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:14.066233 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:29:14.066339 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:14.066670 (+ 331us) server_negotiation.cc:299] Negotiation successful
0806 09:29:14.066928 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":70,"server-negotiator.queue_time_us":62}
I20260806 09:29:14.072001 30704 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 377f07a0afaa4800aa98d87714775150: Bootstrap starting.
I20260806 09:29:14.075219 30704 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 377f07a0afaa4800aa98d87714775150: Neither blocks nor log segments found. Creating new log.
I20260806 09:29:14.076288 30704 log.cc:824] T 00000000000000000000000000000000 P 377f07a0afaa4800aa98d87714775150: Log is configured to *not* fsync() on all Append() calls
I20260806 09:29:14.079064 30704 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 377f07a0afaa4800aa98d87714775150: No bootstrap required, opened a new log
I20260806 09:29:14.086596 30704 raft_consensus.cc:348] T 00000000000000000000000000000000 P 377f07a0afaa4800aa98d87714775150 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cbb5d09bd23462caa29c64dd33fdd25" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 45173 } } peers { permanent_uuid: "377f07a0afaa4800aa98d87714775150" member_type: VOTER last_known_addr { host: "127.20.113.61" port: 41373 } }
I20260806 09:29:14.086930 30704 raft_consensus.cc:374] T 00000000000000000000000000000000 P 377f07a0afaa4800aa98d87714775150 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:29:14.087049 30704 raft_consensus.cc:729] T 00000000000000000000000000000000 P 377f07a0afaa4800aa98d87714775150 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 377f07a0afaa4800aa98d87714775150, State: Initialized, Role: FOLLOWER
I20260806 09:29:14.087602 30704 consensus_queue.cc:260] T 00000000000000000000000000000000 P 377f07a0afaa4800aa98d87714775150 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cbb5d09bd23462caa29c64dd33fdd25" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 45173 } } peers { permanent_uuid: "377f07a0afaa4800aa98d87714775150" member_type: VOTER last_known_addr { host: "127.20.113.61" port: 41373 } }
I20260806 09:29:14.088575 30716 sys_catalog.cc:455] T 00000000000000000000000000000000 P 377f07a0afaa4800aa98d87714775150 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cbb5d09bd23462caa29c64dd33fdd25" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 45173 } } peers { permanent_uuid: "377f07a0afaa4800aa98d87714775150" member_type: VOTER last_known_addr { host: "127.20.113.61" port: 41373 } } }
I20260806 09:29:14.088904 30716 sys_catalog.cc:458] T 00000000000000000000000000000000 P 377f07a0afaa4800aa98d87714775150 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 09:29:14.089581 30704 sys_catalog.cc:564] T 00000000000000000000000000000000 P 377f07a0afaa4800aa98d87714775150 [sys.catalog]: configured and running, proceeding with master startup.
W20260806 09:29:14.099242 30738 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 377f07a0afaa4800aa98d87714775150: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 09:29:14.099393 30738 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 09:29:14.101989 30601 master_runner.cc:428] Detected that this master 377f07a0afaa4800aa98d87714775150 is joining an existing cluster
I20260806 09:29:14.102103 30601 master_runner.cc:432] Initiating AddMaster RPC to add 127.20.113.61:41373
I20260806 09:29:14.131770 30716 raft_consensus.cc:482] T 00000000000000000000000000000000 P 377f07a0afaa4800aa98d87714775150 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 09:29:14.132125 30716 raft_consensus.cc:504] T 00000000000000000000000000000000 P 377f07a0afaa4800aa98d87714775150 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cbb5d09bd23462caa29c64dd33fdd25" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 45173 } } peers { permanent_uuid: "377f07a0afaa4800aa98d87714775150" member_type: VOTER last_known_addr { host: "127.20.113.61" port: 41373 } }
I20260806 09:29:14.133788 30708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:14.108849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59156 (local address 127.20.113.62:45173)
0806 09:29:14.109060 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:14.109066 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:14.116001 (+ 6935us) server_negotiation.cc:407] Connection header received
0806 09:29:14.116235 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:14.116244 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:14.116380 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:14.116485 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:14.118275 (+ 1790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:14.119482 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:14.121869 (+ 2387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:14.122706 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:14.122820 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:14.126030 (+ 3210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:29:14.126041 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:29:14.126049 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:29:14.126102 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:29:14.130003 (+ 3901us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:14.130890 (+ 887us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:14.130898 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:14.130903 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:14.131122 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:14.131674 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:14.131682 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:14.131688 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:14.132087 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:29:14.132227 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:14.133033 (+ 806us) server_negotiation.cc:299] Negotiation successful
0806 09:29:14.133324 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260806 09:29:14.133786 30716 leader_election.cc:288] T 00000000000000000000000000000000 P 377f07a0afaa4800aa98d87714775150 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0cbb5d09bd23462caa29c64dd33fdd25 (127.20.113.62:45173)
I20260806 09:29:14.135046 29933 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "377f07a0afaa4800aa98d87714775150" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cbb5d09bd23462caa29c64dd33fdd25" is_pre_election: true
I20260806 09:29:14.136722 30625 leader_election.cc:302] T 00000000000000000000000000000000 P 377f07a0afaa4800aa98d87714775150 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: 377f07a0afaa4800aa98d87714775150; no voters: 0cbb5d09bd23462caa29c64dd33fdd25
I20260806 09:29:14.137351 30716 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 377f07a0afaa4800aa98d87714775150 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 09:29:14.138549 30747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:14.108381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:45173 (local address 127.0.0.1:59156)
0806 09:29:14.115853 (+ 7472us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:14.115877 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:14.116070 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:14.116734 (+ 664us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:14.116742 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:14.117524 (+ 782us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:29:14.118085 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:14.118102 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:14.119684 (+ 1582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:14.119691 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:14.121663 (+ 1972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:14.121683 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:14.123168 (+ 1485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:14.123176 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:14.123396 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:14.124136 (+ 740us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:29:14.124164 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:29:14.125767 (+ 1603us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:29:14.130178 (+ 4411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:29:14.130186 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:29:14.130192 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:29:14.130753 (+ 561us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:29:14.131305 (+ 552us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:29:14.131316 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:29:14.131322 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:29:14.131523 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:29:14.132351 (+ 828us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:29:14.132358 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:29:14.132655 (+ 297us) client_negotiation.cc:769] Sending connection context
0806 09:29:14.137531 (+ 4876us) client_negotiation.cc:240] Negotiation successful
0806 09:29:14.137951 (+ 420us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7298,"thread_start_us":7140,"threads_started":1}
I20260806 09:29:14.139993 30707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:14.108277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43034 (local address 127.20.113.61:41373)
0806 09:29:14.108610 (+ 333us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:14.108616 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:14.120964 (+ 12348us) server_negotiation.cc:407] Connection header received
0806 09:29:14.121154 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:14.121159 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:14.121232 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:14.121295 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:14.122866 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:14.123984 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:14.126288 (+ 2304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:14.127023 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:14.127120 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:14.129686 (+ 2566us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:29:14.129692 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:29:14.129697 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:29:14.129731 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:29:14.136019 (+ 6288us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:14.136967 (+ 948us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:14.136973 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:14.136978 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:14.137111 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:14.138165 (+ 1054us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:14.138169 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:14.138173 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:14.138599 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:29:14.138751 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:14.139328 (+ 577us) server_negotiation.cc:299] Negotiation successful
0806 09:29:14.139573 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174}
WARNING: no policy specified for oryx/127.20.113.60@KRBTEST.COM; defaulting to no policy
I20260806 09:29:14.141381 30748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:14.112415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.61:41373 (local address 127.0.0.1:43034)
0806 09:29:14.120854 (+ 8439us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:14.120869 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:14.121021 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:14.121503 (+ 482us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:14.121508 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:14.122286 (+ 778us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:29:14.122706 (+ 420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:14.122720 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:14.124242 (+ 1522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:14.124247 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:14.126144 (+ 1897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:14.126159 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:14.127472 (+ 1313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:14.127478 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:14.127641 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:14.128279 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:29:14.128304 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:29:14.129447 (+ 1143us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:29:14.136163 (+ 6716us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:29:14.136168 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:29:14.136171 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:29:14.136844 (+ 673us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:29:14.137851 (+ 1007us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:29:14.137856 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:29:14.137859 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:29:14.138044 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:29:14.138844 (+ 800us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:29:14.138850 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:29:14.139023 (+ 173us) client_negotiation.cc:769] Sending connection context
0806 09:29:14.140461 (+ 1438us) client_negotiation.cc:240] Negotiation successful
0806 09:29:14.140777 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8317,"thread_start_us":8090,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.113.60@KRBTEST.COM" created.
Aug 06 09:29:14 dist-test-slave-45dx krb5kdc[29852](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008551, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.113.62@KRBTEST.COM for oryx/127.20.113.61@KRBTEST.COM
I20260806 09:29:14.175231 30707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:14.145920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43048 (local address 127.20.113.61:41373)
0806 09:29:14.146107 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:14.146115 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:14.155983 (+ 9868us) server_negotiation.cc:407] Connection header received
0806 09:29:14.156465 (+ 482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:14.156471 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:14.156573 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:14.156709 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:14.158291 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:14.159499 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:14.161735 (+ 2236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:14.162511 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:14.162669 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:14.167476 (+ 4807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:29:14.167485 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:29:14.167492 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:29:14.167540 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:29:14.171670 (+ 4130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:14.172661 (+ 991us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:14.172667 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:14.172670 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:14.172883 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:14.173373 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:14.173380 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:14.173386 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:14.173856 (+ 470us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:29:14.174030 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:14.174668 (+ 638us) server_negotiation.cc:299] Negotiation successful
0806 09:29:14.174848 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 09:29:14.175335 30751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:14.145900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.61:41373 (local address 127.0.0.1:43048)
0806 09:29:14.155813 (+ 9913us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:14.155874 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:14.156168 (+ 294us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:14.156786 (+ 618us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:14.156797 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:14.157786 (+ 989us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:29:14.158113 (+ 327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:14.158131 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:14.159668 (+ 1537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:14.159676 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:14.161498 (+ 1822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:14.161521 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:14.162713 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:14.162722 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:14.162998 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:14.163904 (+ 906us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:29:14.163933 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:29:14.167249 (+ 3316us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:29:14.171820 (+ 4571us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:29:14.171833 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:29:14.171842 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:29:14.172493 (+ 651us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:29:14.173019 (+ 526us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:29:14.173024 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:29:14.173028 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:29:14.173210 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:29:14.174081 (+ 871us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:29:14.174091 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:29:14.174305 (+ 214us) client_negotiation.cc:769] Sending connection context
0806 09:29:14.174553 (+ 248us) client_negotiation.cc:240] Negotiation successful
0806 09:29:14.174889 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9584,"thread_start_us":9418,"threads_started":1}
I20260806 09:29:14.177565 29915 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.20.113.61:41373: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "377f07a0afaa4800aa98d87714775150"
member_type: NON_VOTER
last_known_addr {
host: "127.20.113.61"
port: 41373
}
attrs {
promote: true
}
}
dest_uuid: "0cbb5d09bd23462caa29c64dd33fdd25"
cas_config_opid_index: -1
I20260806 09:29:14.178665 29915 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0cbb5d09bd23462caa29c64dd33fdd25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0cbb5d09bd23462caa29c64dd33fdd25" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 45173 } } peers { permanent_uuid: "377f07a0afaa4800aa98d87714775150" member_type: NON_VOTER last_known_addr { host: "127.20.113.61" port: 41373 } attrs { promote: true } }
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.113.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/krb5kdc/oryx_127.20.113.60.keytab.
Entry for principal oryx/127.20.113.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/krb5kdc/oryx_127.20.113.60.keytab.
I20260806 09:29:14.182549 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.113.60:42483
--webserver_interface=127.20.113.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/krb5kdc/oryx_127.20.113.60.keytab
--principal=oryx/127.20.113.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:41701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.113.62:45173,127.20.113.61:41373,127.20.113.60:42483
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 09:29:14.186798 30758 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0cbb5d09bd23462caa29c64dd33fdd25 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 377f07a0afaa4800aa98d87714775150 (127.20.113.61) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0cbb5d09bd23462caa29c64dd33fdd25" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 45173 } } peers { permanent_uuid: "377f07a0afaa4800aa98d87714775150" member_type: NON_VOTER last_known_addr { host: "127.20.113.61" port: 41373 } attrs { promote: true } } }
I20260806 09:29:14.189023 30663 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 377f07a0afaa4800aa98d87714775150 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:29:14.191505 30758 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.20.113.61:41373
I20260806 09:29:14.193005 30663 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 377f07a0afaa4800aa98d87714775150 [term 1 FOLLOWER]: Refusing update from remote peer 0cbb5d09bd23462caa29c64dd33fdd25: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260806 09:29:14.193552 30601 master.cc:537] Master@127.20.113.61:41373 shutting down...
I20260806 09:29:14.195879 30757 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0cbb5d09bd23462caa29c64dd33fdd25 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "0cbb5d09bd23462caa29c64dd33fdd25" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0cbb5d09bd23462caa29c64dd33fdd25" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 45173 } } peers { permanent_uuid: "377f07a0afaa4800aa98d87714775150" member_type: NON_VOTER last_known_addr { host: "127.20.113.61" port: 41373 } attrs { promote: true } } }
I20260806 09:29:14.196445 30757 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0cbb5d09bd23462caa29c64dd33fdd25 [sys.catalog]: This master's current role is: LEADER
I20260806 09:29:14.197405 30760 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0cbb5d09bd23462caa29c64dd33fdd25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "377f07a0afaa4800aa98d87714775150" member_type: NON_VOTER last_known_addr { host: "127.20.113.61" port: 41373 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 09:29:14.204034 29895 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260806 09:29:14.209762 30601 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 377f07a0afaa4800aa98d87714775150 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 09:29:14.210165 30601 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 377f07a0afaa4800aa98d87714775150 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 09:29:14.210386 30601 tablet_replica.cc:330] T 00000000000000000000000000000000 P 377f07a0afaa4800aa98d87714775150: stopping tablet replica
W20260806 09:29:14.215780 29895 consensus_peers.cc:487] T 00000000000000000000000000000000 P 0cbb5d09bd23462caa29c64dd33fdd25 -> Peer 377f07a0afaa4800aa98d87714775150 (127.20.113.61:41373): Couldn't send request to peer 377f07a0afaa4800aa98d87714775150. Status: Network error: recv got EOF from unknown peer (error 108). This is attempt 1: this message will repeat every 5th retry.
I20260806 09:29:14.228111 30601 master.cc:552] Master@127.20.113.61:41373 shutdown complete.
I20260806 09:29:14.228295 30601 master_runner.cc:305] Clearing existing system tablet
I20260806 09:29:14.228915 30601 fs_manager.cc:359] Metadata directory not provided
I20260806 09:29:14.229032 30601 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:29:14.232618 30601 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.001s
I20260806 09:29:14.234480 30766 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:29:14.234866 30601 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:29:14.235260 30601 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260806 09:29:14.235368 30601 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/master-1/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/master-1/wal
uuid: "377f07a0afaa4800aa98d87714775150"
format_stamp: "Formatted at 2026-08-06 09:29:13 on dist-test-slave-45dx"
server_key: "3ba94b12271f82bfd9e3dfd83172fb08"
server_key_iv: "8fb0b78efbd3af269b272eba60edbb7a"
server_key_version: "encryptionkey@0"
I20260806 09:29:14.240620 30601 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 377f07a0afaa4800aa98d87714775150: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 09:29:14.248049 30601 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 377f07a0afaa4800aa98d87714775150: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260806 09:29:14.248248 30601 log.cc:1196] T 00000000000000000000000000000000 P 377f07a0afaa4800aa98d87714775150: Deleting WAL directory at /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260806 09:29:14.248731 30601 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 377f07a0afaa4800aa98d87714775150: Deleting consensus metadata
I20260806 09:29:14.249038 30601 master_runner.cc:315] Copying system tablet from 127.20.113.62:45173
I20260806 09:29:14.251739 30601 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 377f07a0afaa4800aa98d87714775150: tablet copy: Beginning tablet copy session from remote peer at address 127.20.113.62:45173
I20260806 09:29:14.269549 30773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:14.252553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:45173 (local address 127.0.0.1:59172)
0806 09:29:14.253280 (+ 727us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:14.253306 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:14.253494 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:14.254009 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:14.254015 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:14.254586 (+ 571us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:29:14.254894 (+ 308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:14.254909 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:14.256437 (+ 1528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:14.256444 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:14.258078 (+ 1634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:14.258095 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:14.259236 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:14.259242 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:14.259461 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:14.260281 (+ 820us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:29:14.260328 (+ 47us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:29:14.261586 (+ 1258us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:29:14.265915 (+ 4329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:29:14.265921 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:29:14.265925 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:29:14.266471 (+ 546us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:29:14.266957 (+ 486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:29:14.266963 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:29:14.266966 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:29:14.267157 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:29:14.267911 (+ 754us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:29:14.267916 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:29:14.268121 (+ 205us) client_negotiation.cc:769] Sending connection context
0806 09:29:14.268350 (+ 229us) client_negotiation.cc:240] Negotiation successful
0806 09:29:14.268848 (+ 498us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":575,"thread_start_us":435,"threads_started":1}
I20260806 09:29:14.270686 30708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:14.252589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59172 (local address 127.20.113.62:45173)
0806 09:29:14.252783 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:14.252789 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:14.253431 (+ 642us) server_negotiation.cc:407] Connection header received
0806 09:29:14.253719 (+ 288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:14.253725 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:14.253830 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:14.253960 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:14.255078 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:14.256233 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:14.258277 (+ 2044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:14.259044 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:14.259190 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:14.261837 (+ 2647us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:29:14.261844 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:29:14.261849 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:29:14.261894 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:29:14.265750 (+ 3856us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:14.266624 (+ 874us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:14.266630 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:14.266634 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:14.266806 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:14.267297 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:14.267302 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:14.267306 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:14.267695 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:29:14.267848 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:14.270043 (+ 2195us) server_negotiation.cc:299] Negotiation successful
0806 09:29:14.270304 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 09:29:14.272689 29950 tablet_copy_service.cc:140] P 0cbb5d09bd23462caa29c64dd33fdd25: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 377f07a0afaa4800aa98d87714775150 ({username='oryx', principal='oryx/127.20.113.61@KRBTEST.COM'} at 127.0.0.1:59172)
I20260806 09:29:14.272995 29950 tablet_copy_service.cc:161] P 0cbb5d09bd23462caa29c64dd33fdd25: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 377f07a0afaa4800aa98d87714775150 at {username='oryx', principal='oryx/127.20.113.61@KRBTEST.COM'} at 127.0.0.1:59172: session id = 377f07a0afaa4800aa98d87714775150-00000000000000000000000000000000
I20260806 09:29:14.276566 29950 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 0cbb5d09bd23462caa29c64dd33fdd25: Tablet Copy: opened 0 blocks and 1 log segments
I20260806 09:29:14.279116 30601 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260806 09:29:14.286249 30601 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 377f07a0afaa4800aa98d87714775150: tablet copy: Starting download of 0 data blocks...
I20260806 09:29:14.286579 30601 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 377f07a0afaa4800aa98d87714775150: tablet copy: Starting download of 1 WAL segments...
I20260806 09:29:14.289553 30601 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 377f07a0afaa4800aa98d87714775150: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260806 09:29:14.294327 29950 tablet_copy_service.cc:342] P 0cbb5d09bd23462caa29c64dd33fdd25: Request end of tablet copy session 377f07a0afaa4800aa98d87714775150-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.20.113.61@KRBTEST.COM'} at 127.0.0.1:59172
I20260806 09:29:14.294533 29950 tablet_copy_service.cc:433] P 0cbb5d09bd23462caa29c64dd33fdd25: ending tablet copy session 377f07a0afaa4800aa98d87714775150-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 377f07a0afaa4800aa98d87714775150
I20260806 09:29:14.307521 30601 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:29:14.314105 30774 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:29:14.314744 30775 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:29:14.324579 30601 server_base.cc:1034] running on GCE node
W20260806 09:29:14.325627 30777 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:29:14.326160 30601 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:29:14.326963 30601 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:29:14.328267 30601 hybrid_clock.cc:648] HybridClock initialized: now 1786008554328194 us; error 49 us; skew 500 ppm
I20260806 09:29:14.328547 30601 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:29:14 dist-test-slave-45dx krb5kdc[29852](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008554, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.113.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:29:14.331032 30601 init.cc:377] Logged in from keytab as oryx/127.20.113.61@KRBTEST.COM (short username oryx)
I20260806 09:29:14.332453 30601 webserver.cc:460] Webserver started at http://127.20.113.61:38841/ using document root <none> and password file <none>
I20260806 09:29:14.332912 30601 fs_manager.cc:359] Metadata directory not provided
I20260806 09:29:14.333004 30601 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:29:14.336433 30601 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260806 09:29:14.338840 30787 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:29:14.339346 30601 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 09:29:14.339476 30601 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/master-1/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/master-1/wal
uuid: "377f07a0afaa4800aa98d87714775150"
format_stamp: "Formatted at 2026-08-06 09:29:13 on dist-test-slave-45dx"
server_key: "3ba94b12271f82bfd9e3dfd83172fb08"
server_key_iv: "8fb0b78efbd3af269b272eba60edbb7a"
server_key_version: "encryptionkey@0"
I20260806 09:29:14.339640 30601 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:29:14.354249 30601 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:29:14.358690 30601 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:29:14.401507 30601 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.61:41373
I20260806 09:29:14.401554 30866 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.61:41373 every 8 connection(s)
I20260806 09:29:14.402976 30601 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/master-1/data/info.pb
I20260806 09:29:14.409927 30867 sys_catalog.cc:263] Verifying existing consensus state
I20260806 09:29:14.414587 30867 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 377f07a0afaa4800aa98d87714775150: Bootstrap starting.
I20260806 09:29:14.434190 30867 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 377f07a0afaa4800aa98d87714775150: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 09:29:14.434796 30867 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 377f07a0afaa4800aa98d87714775150: Bootstrap complete.
I20260806 09:29:14.436201 30867 raft_consensus.cc:348] T 00000000000000000000000000000000 P 377f07a0afaa4800aa98d87714775150 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0cbb5d09bd23462caa29c64dd33fdd25" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 45173 } } peers { permanent_uuid: "377f07a0afaa4800aa98d87714775150" member_type: NON_VOTER last_known_addr { host: "127.20.113.61" port: 41373 } attrs { promote: true } }
I20260806 09:29:14.436702 30867 raft_consensus.cc:729] T 00000000000000000000000000000000 P 377f07a0afaa4800aa98d87714775150 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 377f07a0afaa4800aa98d87714775150, State: Initialized, Role: LEARNER
I20260806 09:29:14.437170 30867 consensus_queue.cc:260] T 00000000000000000000000000000000 P 377f07a0afaa4800aa98d87714775150 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0cbb5d09bd23462caa29c64dd33fdd25" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 45173 } } peers { permanent_uuid: "377f07a0afaa4800aa98d87714775150" member_type: NON_VOTER last_known_addr { host: "127.20.113.61" port: 41373 } attrs { promote: true } }
I20260806 09:29:14.438549 30867 sys_catalog.cc:564] T 00000000000000000000000000000000 P 377f07a0afaa4800aa98d87714775150 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 09:29:14.438257 30873 sys_catalog.cc:455] T 00000000000000000000000000000000 P 377f07a0afaa4800aa98d87714775150 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0cbb5d09bd23462caa29c64dd33fdd25" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 45173 } } peers { permanent_uuid: "377f07a0afaa4800aa98d87714775150" member_type: NON_VOTER last_known_addr { host: "127.20.113.61" port: 41373 } attrs { promote: true } } }
I20260806 09:29:14.438903 30873 sys_catalog.cc:458] T 00000000000000000000000000000000 P 377f07a0afaa4800aa98d87714775150 [sys.catalog]: This master's current role is: LEARNER
I20260806 09:29:14.447712 30888 catalog_manager.cc:1259] Loaded cluster ID: e50f8b022b59446eaf1c3aaefb8eb3e7
I20260806 09:29:14.447831 30888 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 377f07a0afaa4800aa98d87714775150: loading cluster ID for follower catalog manager: success
I20260806 09:29:14.450502 30888 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 377f07a0afaa4800aa98d87714775150: acquiring CA information for follower catalog manager: success
I20260806 09:29:14.452520 30888 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 377f07a0afaa4800aa98d87714775150: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 09:29:14.488826 30591 heartbeater.cc:503] Master 127.20.113.62:45173 was elected leader, sending a full tablet report...
I20260806 09:29:14.625849 30751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:14.616962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.61:41373 (local address 127.0.0.1:43054)
0806 09:29:14.617251 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:14.617291 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:14.617437 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:14.618541 (+ 1104us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:14.618546 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:14.618580 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:29:14.618820 (+ 240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:14.618831 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:14.620836 (+ 2005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:14.620844 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:14.623012 (+ 2168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:14.623030 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:14.624797 (+ 1767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:14.624806 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:14.625092 (+ 286us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:14.625193 (+ 101us) client_negotiation.cc:769] Sending connection context
0806 09:29:14.625305 (+ 112us) client_negotiation.cc:240] Negotiation successful
0806 09:29:14.625554 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146}
I20260806 09:29:14.625897 30889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:14.617134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43054 (local address 127.20.113.61:41373)
0806 09:29:14.618122 (+ 988us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:14.618129 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:14.618152 (+ 23us) server_negotiation.cc:407] Connection header received
0806 09:29:14.618230 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:14.618235 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:14.618324 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:14.618458 (+ 134us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:29:14.619011 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:14.620619 (+ 1608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:14.623196 (+ 2577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:14.624603 (+ 1407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:14.624736 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:14.624976 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:14.625343 (+ 367us) server_negotiation.cc:299] Negotiation successful
0806 09:29:14.625505 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":848,"thread_start_us":617,"threads_started":1}
I20260806 09:29:14.627537 30827 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 377f07a0afaa4800aa98d87714775150 [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.5] Dedup: 1.5->[]
I20260806 09:29:14.628211 30873 sys_catalog.cc:455] T 00000000000000000000000000000000 P 377f07a0afaa4800aa98d87714775150 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0cbb5d09bd23462caa29c64dd33fdd25. Latest consensus state: current_term: 1 leader_uuid: "0cbb5d09bd23462caa29c64dd33fdd25" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0cbb5d09bd23462caa29c64dd33fdd25" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 45173 } } peers { permanent_uuid: "377f07a0afaa4800aa98d87714775150" member_type: NON_VOTER last_known_addr { host: "127.20.113.61" port: 41373 } attrs { promote: true } } }
I20260806 09:29:14.628530 30873 sys_catalog.cc:458] T 00000000000000000000000000000000 P 377f07a0afaa4800aa98d87714775150 [sys.catalog]: This master's current role is: LEARNER
W20260806 09:29:14.686844 30761 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:29:14.687171 30761 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:29:14.687325 30761 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:29:14.695140 30761 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 09:29:14.695307 30761 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:29:14.695428 30761 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:29:14.695533 30761 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 09:29:14.695608 30761 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 09:29:14.706154 30761 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:41701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.20.113.62:45173,127.20.113.61:41373,127.20.113.60:42483
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/krb5kdc/oryx_127.20.113.60.keytab
--principal=oryx/127.20.113.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.113.60:42483
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.20.113.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:29:14.707571 30761 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:29:14.709072 30761 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:29:14.721202 30898 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:29:14.721153 30899 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:29:14.721246 30901 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:29:14.722532 30761 server_base.cc:1034] running on GCE node
I20260806 09:29:14.723501 30761 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:29:14.725168 30761 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:29:14.726377 30761 hybrid_clock.cc:648] HybridClock initialized: now 1786008554726294 us; error 71 us; skew 500 ppm
I20260806 09:29:14.726742 30761 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:29:14 dist-test-slave-45dx krb5kdc[29852](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008554, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.113.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:29:14.730101 30761 init.cc:377] Logged in from keytab as oryx/127.20.113.60@KRBTEST.COM (short username oryx)
I20260806 09:29:14.731721 30761 webserver.cc:460] Webserver started at http://127.20.113.60:34813/ using document root <none> and password file <none>
I20260806 09:29:14.732491 30761 fs_manager.cc:359] Metadata directory not provided
I20260806 09:29:14.732643 30761 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:29:14.733013 30761 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:29:14.736853 30761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/master-2/data/instance:
uuid: "e8ec8de97a86401e948b6ffb6a72e7d1"
format_stamp: "Formatted at 2026-08-06 09:29:14 on dist-test-slave-45dx"
server_key: "349707703198dbc3a5c4780f075bd1b7"
server_key_iv: "b85360db13c32a6d7b34c4723b64ad6b"
server_key_version: "encryptionkey@0"
I20260806 09:29:14.737962 30761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/master-2/wal/instance:
uuid: "e8ec8de97a86401e948b6ffb6a72e7d1"
format_stamp: "Formatted at 2026-08-06 09:29:14 on dist-test-slave-45dx"
server_key: "349707703198dbc3a5c4780f075bd1b7"
server_key_iv: "b85360db13c32a6d7b34c4723b64ad6b"
server_key_version: "encryptionkey@0"
I20260806 09:29:14.745595 30761 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.008s sys 0.000s
I20260806 09:29:14.749289 30911 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:29:14.750739 30761 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 09:29:14.750919 30761 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/master-2/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/master-2/wal
uuid: "e8ec8de97a86401e948b6ffb6a72e7d1"
format_stamp: "Formatted at 2026-08-06 09:29:14 on dist-test-slave-45dx"
server_key: "349707703198dbc3a5c4780f075bd1b7"
server_key_iv: "b85360db13c32a6d7b34c4723b64ad6b"
server_key_version: "encryptionkey@0"
I20260806 09:29:14.751101 30761 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:29:14.774456 30761 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:29:14.778880 30761 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:29:14.779219 30761 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:29:14.799185 30761 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.60:42483
I20260806 09:29:14.799315 30988 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.60:42483 every 8 connection(s)
I20260806 09:29:14.800638 30761 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/master-2/data/info.pb
I20260806 09:29:14.802421 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 30761
I20260806 09:29:14.806501 30990 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:29:14.813603 30990 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.113.62" port: 45173 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 09:29:14 dist-test-slave-45dx krb5kdc[29852](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008551, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.20.113.60@KRBTEST.COM
Aug 06 09:29:14 dist-test-slave-45dx krb5kdc[29852](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008554, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.113.60@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM
I20260806 09:29:14.826159 30993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:14.804642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60226 (local address 127.20.113.60:42483)
0806 09:29:14.805322 (+ 680us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:14.805334 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:14.805375 (+ 41us) server_negotiation.cc:407] Connection header received
0806 09:29:14.807060 (+ 1685us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:14.807082 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:14.807444 (+ 362us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:14.807819 (+ 375us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:14.809540 (+ 1721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:14.810935 (+ 1395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:14.812236 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:14.812960 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:14.813185 (+ 225us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:14.818207 (+ 5022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:29:14.818215 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:29:14.818228 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:29:14.818271 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:29:14.821383 (+ 3112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:14.822124 (+ 741us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:14.822130 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:14.822137 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:14.822405 (+ 268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:14.822905 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:14.822913 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:14.822918 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:14.823649 (+ 731us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:29:14.823985 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:14.824507 (+ 522us) server_negotiation.cc:299] Negotiation successful
0806 09:29:14.824893 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":488,"thread_start_us":295,"threads_started":1}
I20260806 09:29:14.832196 30995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:14.816092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:45173 (local address 127.0.0.1:59186)
0806 09:29:14.817080 (+ 988us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:14.817163 (+ 83us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:14.817333 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:14.817769 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:14.817777 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:14.818502 (+ 725us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:29:14.818826 (+ 324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:14.818835 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:14.819833 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:14.819839 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:14.821071 (+ 1232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:14.821109 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:14.822168 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:14.822177 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:14.822381 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:14.823033 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:29:14.823061 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:29:14.825862 (+ 2801us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:29:14.829497 (+ 3635us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:29:14.829528 (+ 31us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:29:14.829535 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:29:14.829982 (+ 447us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:29:14.830373 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:29:14.830378 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:29:14.830382 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:29:14.830527 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:29:14.831067 (+ 540us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:29:14.831074 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:29:14.831261 (+ 187us) client_negotiation.cc:769] Sending connection context
0806 09:29:14.831440 (+ 179us) client_negotiation.cc:240] Negotiation successful
0806 09:29:14.831753 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":783,"thread_start_us":676,"threads_started":1}
I20260806 09:29:14.832229 30994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:14.816136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59186 (local address 127.20.113.62:45173)
0806 09:29:14.816617 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:14.816622 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:14.817260 (+ 638us) server_negotiation.cc:407] Connection header received
0806 09:29:14.817489 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:14.817494 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:14.817574 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:14.817692 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:14.818992 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:14.819689 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:14.821270 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:14.821895 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:14.822054 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:14.826072 (+ 4018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:29:14.826077 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:29:14.826080 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:29:14.826113 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:29:14.829357 (+ 3244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:14.830129 (+ 772us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:14.830133 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:14.830135 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:14.830256 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:14.830657 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:14.830661 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:14.830663 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:14.830931 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:29:14.831090 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:14.831559 (+ 469us) server_negotiation.cc:299] Negotiation successful
0806 09:29:14.831792 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":391,"thread_start_us":275,"threads_started":1}
I20260806 09:29:14.833540 30990 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.113.61" port: 41373 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 09:29:14 dist-test-slave-45dx krb5kdc[29852](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008554, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.113.60@KRBTEST.COM for oryx/127.20.113.61@KRBTEST.COM
I20260806 09:29:14.846521 30995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:14.834199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.61:41373 (local address 127.0.0.1:43058)
0806 09:29:14.834379 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:14.834395 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:14.834471 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:14.834842 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:14.834846 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:14.835143 (+ 297us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:29:14.835309 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:14.835317 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:14.836284 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:14.836288 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:14.837288 (+ 1000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:14.837301 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:14.837971 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:14.837975 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:14.838122 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:14.838725 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:29:14.838741 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:29:14.840887 (+ 2146us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:29:14.843984 (+ 3097us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:29:14.843989 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:29:14.843993 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:29:14.844448 (+ 455us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:29:14.844839 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:29:14.844844 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:29:14.844848 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:29:14.845001 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:29:14.845539 (+ 538us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:29:14.845543 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:29:14.845703 (+ 160us) client_negotiation.cc:769] Sending connection context
0806 09:29:14.845840 (+ 137us) client_negotiation.cc:240] Negotiation successful
0806 09:29:14.846159 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64}
I20260806 09:29:14.846546 30889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:14.834412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43058 (local address 127.20.113.61:41373)
0806 09:29:14.834584 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:14.834588 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:14.834603 (+ 15us) server_negotiation.cc:407] Connection header received
0806 09:29:14.834652 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:14.834655 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:14.834714 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:14.834793 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:14.835461 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:14.836152 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:14.837416 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:14.837831 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:14.837924 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:14.841116 (+ 3192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:29:14.841120 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:29:14.841123 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:29:14.841155 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:29:14.843845 (+ 2690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:14.844569 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:14.844575 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:14.844578 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:14.844720 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:14.845133 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:14.845136 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:14.845138 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:14.845403 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:29:14.845581 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:14.845950 (+ 369us) server_negotiation.cc:299] Negotiation successful
0806 09:29:14.846189 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 09:29:14.847599 30990 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.113.60" port: 42483 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 09:29:14 dist-test-slave-45dx krb5kdc[29852](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008554, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.113.60@KRBTEST.COM for oryx/127.20.113.60@KRBTEST.COM
I20260806 09:29:14.859684 30995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:14.848236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.60:42483 (local address 127.0.0.1:60236)
0806 09:29:14.848528 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:14.848548 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:14.848644 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:14.848910 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:14.848915 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:14.849268 (+ 353us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:29:14.849424 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:14.849432 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:14.850377 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:14.850384 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:14.851187 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:14.851197 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:14.851875 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:14.851881 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:14.852081 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:14.852662 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:29:14.852678 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:29:14.854535 (+ 1857us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:29:14.857326 (+ 2791us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:29:14.857332 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:29:14.857336 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:29:14.857679 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:29:14.858048 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:29:14.858053 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:29:14.858056 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:29:14.858205 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:29:14.858694 (+ 489us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:29:14.858698 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:29:14.858845 (+ 147us) client_negotiation.cc:769] Sending connection context
0806 09:29:14.858977 (+ 132us) client_negotiation.cc:240] Negotiation successful
0806 09:29:14.859232 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119}
I20260806 09:29:14.859748 30993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:14.848496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60236 (local address 127.20.113.60:42483)
0806 09:29:14.848658 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:14.848663 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:14.848680 (+ 17us) server_negotiation.cc:407] Connection header received
0806 09:29:14.848727 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:14.848733 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:14.848793 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:14.848900 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:14.849566 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:14.850253 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:14.851320 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:14.851715 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:14.851839 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:14.854715 (+ 2876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:29:14.854727 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:29:14.854730 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:29:14.854757 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:29:14.857184 (+ 2427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:14.857793 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:14.857800 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:14.857804 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:14.857938 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:14.858310 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:14.858316 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:14.858320 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:14.858583 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:29:14.858728 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:14.859063 (+ 335us) server_negotiation.cc:299] Negotiation successful
0806 09:29:14.859390 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":109,"server-negotiator.queue_time_us":75}
I20260806 09:29:14.864570 30990 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8ec8de97a86401e948b6ffb6a72e7d1: Bootstrap starting.
I20260806 09:29:14.867787 30990 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e8ec8de97a86401e948b6ffb6a72e7d1: Neither blocks nor log segments found. Creating new log.
I20260806 09:29:14.868858 30990 log.cc:824] T 00000000000000000000000000000000 P e8ec8de97a86401e948b6ffb6a72e7d1: Log is configured to *not* fsync() on all Append() calls
I20260806 09:29:14.871469 30990 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8ec8de97a86401e948b6ffb6a72e7d1: No bootstrap required, opened a new log
I20260806 09:29:14.878965 30990 raft_consensus.cc:348] T 00000000000000000000000000000000 P e8ec8de97a86401e948b6ffb6a72e7d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cbb5d09bd23462caa29c64dd33fdd25" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 45173 } } peers { permanent_uuid: "377f07a0afaa4800aa98d87714775150" member_type: VOTER last_known_addr { host: "127.20.113.61" port: 41373 } } peers { permanent_uuid: "e8ec8de97a86401e948b6ffb6a72e7d1" member_type: VOTER last_known_addr { host: "127.20.113.60" port: 42483 } }
I20260806 09:29:14.879309 30990 raft_consensus.cc:374] T 00000000000000000000000000000000 P e8ec8de97a86401e948b6ffb6a72e7d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:29:14.879455 30990 raft_consensus.cc:729] T 00000000000000000000000000000000 P e8ec8de97a86401e948b6ffb6a72e7d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8ec8de97a86401e948b6ffb6a72e7d1, State: Initialized, Role: FOLLOWER
I20260806 09:29:14.880102 30990 consensus_queue.cc:260] T 00000000000000000000000000000000 P e8ec8de97a86401e948b6ffb6a72e7d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cbb5d09bd23462caa29c64dd33fdd25" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 45173 } } peers { permanent_uuid: "377f07a0afaa4800aa98d87714775150" member_type: VOTER last_known_addr { host: "127.20.113.61" port: 41373 } } peers { permanent_uuid: "e8ec8de97a86401e948b6ffb6a72e7d1" member_type: VOTER last_known_addr { host: "127.20.113.60" port: 42483 } }
I20260806 09:29:14.881248 30997 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8ec8de97a86401e948b6ffb6a72e7d1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cbb5d09bd23462caa29c64dd33fdd25" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 45173 } } peers { permanent_uuid: "377f07a0afaa4800aa98d87714775150" member_type: VOTER last_known_addr { host: "127.20.113.61" port: 41373 } } peers { permanent_uuid: "e8ec8de97a86401e948b6ffb6a72e7d1" member_type: VOTER last_known_addr { host: "127.20.113.60" port: 42483 } } }
I20260806 09:29:14.881630 30997 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8ec8de97a86401e948b6ffb6a72e7d1 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 09:29:14.882337 30990 sys_catalog.cc:564] T 00000000000000000000000000000000 P e8ec8de97a86401e948b6ffb6a72e7d1 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 09:29:14.885128 30997 raft_consensus.cc:482] T 00000000000000000000000000000000 P e8ec8de97a86401e948b6ffb6a72e7d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 09:29:14.885411 30997 raft_consensus.cc:504] T 00000000000000000000000000000000 P e8ec8de97a86401e948b6ffb6a72e7d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cbb5d09bd23462caa29c64dd33fdd25" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 45173 } } peers { permanent_uuid: "377f07a0afaa4800aa98d87714775150" member_type: VOTER last_known_addr { host: "127.20.113.61" port: 41373 } } peers { permanent_uuid: "e8ec8de97a86401e948b6ffb6a72e7d1" member_type: VOTER last_known_addr { host: "127.20.113.60" port: 42483 } }
I20260806 09:29:14.887130 30997 leader_election.cc:288] T 00000000000000000000000000000000 P e8ec8de97a86401e948b6ffb6a72e7d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0cbb5d09bd23462caa29c64dd33fdd25 (127.20.113.62:45173), 377f07a0afaa4800aa98d87714775150 (127.20.113.61:41373)
I20260806 09:29:14.887758 29933 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e8ec8de97a86401e948b6ffb6a72e7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cbb5d09bd23462caa29c64dd33fdd25" is_pre_election: true
I20260806 09:29:14.888356 30827 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e8ec8de97a86401e948b6ffb6a72e7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "377f07a0afaa4800aa98d87714775150" is_pre_election: true
I20260806 09:29:14.889281 30915 leader_election.cc:302] T 00000000000000000000000000000000 P e8ec8de97a86401e948b6ffb6a72e7d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8ec8de97a86401e948b6ffb6a72e7d1; no voters: 0cbb5d09bd23462caa29c64dd33fdd25, 377f07a0afaa4800aa98d87714775150
I20260806 09:29:14.889797 30997 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e8ec8de97a86401e948b6ffb6a72e7d1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260806 09:29:14.892036 31013 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e8ec8de97a86401e948b6ffb6a72e7d1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 09:29:14.892186 31013 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 09:29:14.897215 30761 master_runner.cc:428] Detected that this master e8ec8de97a86401e948b6ffb6a72e7d1 is joining an existing cluster
I20260806 09:29:14.897351 30761 master_runner.cc:432] Initiating AddMaster RPC to add 127.20.113.60:42483
I20260806 09:29:14.917701 30994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:14.902119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59190 (local address 127.20.113.62:45173)
0806 09:29:14.902339 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:14.902347 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:14.904972 (+ 2625us) server_negotiation.cc:407] Connection header received
0806 09:29:14.905179 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:14.905188 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:14.905251 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:14.905356 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:14.906484 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:14.907208 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:14.908830 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:14.909425 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:14.909557 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:14.911594 (+ 2037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:29:14.911600 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:29:14.911605 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:29:14.911646 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:29:14.914326 (+ 2680us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:14.915173 (+ 847us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:14.915179 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:14.915184 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:14.915347 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:14.915834 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:14.915839 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:14.915842 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:14.916130 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:29:14.916312 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:14.916845 (+ 533us) server_negotiation.cc:299] Negotiation successful
0806 09:29:14.917309 (+ 464us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260806 09:29:14.919343 31023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:14.901566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:45173 (local address 127.0.0.1:59190)
0806 09:29:14.904852 (+ 3286us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:14.904874 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:14.905038 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:14.905382 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:14.905387 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:14.906010 (+ 623us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:29:14.906335 (+ 325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:14.906344 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:14.907347 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:14.907354 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:14.908693 (+ 1339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:14.908705 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:14.909640 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:14.909650 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:14.909839 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:14.910572 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:29:14.910590 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:29:14.911394 (+ 804us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:29:14.914472 (+ 3078us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:29:14.914479 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:29:14.914483 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:29:14.915003 (+ 520us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:29:14.915514 (+ 511us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:29:14.915519 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:29:14.915522 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:29:14.915719 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:29:14.916255 (+ 536us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:29:14.916260 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:29:14.916529 (+ 269us) client_negotiation.cc:769] Sending connection context
0806 09:29:14.916736 (+ 207us) client_negotiation.cc:240] Negotiation successful
0806 09:29:14.918862 (+ 2126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3126,"thread_start_us":2967,"threads_started":1}
I20260806 09:29:14.933068 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 30010
I20260806 09:29:14.940092 31023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:14.917114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.60:42483 (local address 127.0.0.1:60244)
0806 09:29:14.921515 (+ 4401us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:14.921533 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:14.921654 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:14.922088 (+ 434us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:14.922093 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:14.922588 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:29:14.922831 (+ 243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:14.922843 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:14.925851 (+ 3008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:14.925855 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:14.927303 (+ 1448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:14.927320 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:14.928631 (+ 1311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:14.928637 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:14.928819 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:14.929407 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:29:14.929430 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:29:14.930220 (+ 790us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:29:14.936917 (+ 6697us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:29:14.936925 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:29:14.936932 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:29:14.937482 (+ 550us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:29:14.937937 (+ 455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:29:14.937942 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:29:14.937948 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:29:14.938116 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:29:14.938766 (+ 650us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:29:14.938774 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:29:14.938954 (+ 180us) client_negotiation.cc:769] Sending connection context
0806 09:29:14.939118 (+ 164us) client_negotiation.cc:240] Negotiation successful
0806 09:29:14.939668 (+ 550us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4305,"mutex_wait_us":1,"thread_start_us":14970,"threads_started":1}
I20260806 09:29:14.940200 30993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:14.917090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60244 (local address 127.20.113.60:42483)
0806 09:29:14.917273 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:14.917278 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:14.921588 (+ 4310us) server_negotiation.cc:407] Connection header received
0806 09:29:14.921790 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:14.921795 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:14.921854 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:14.921927 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:14.924472 (+ 2545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:14.925638 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:14.927438 (+ 1800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:14.928215 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:14.928335 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:14.930407 (+ 2072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:29:14.930413 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:29:14.930418 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:29:14.930452 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:29:14.935996 (+ 5544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:14.937639 (+ 1643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:14.937645 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:14.937650 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:14.937788 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:14.938240 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:14.938245 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:14.938249 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:14.938621 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:29:14.939361 (+ 740us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:14.939501 (+ 140us) server_negotiation.cc:299] Negotiation successful
0806 09:29:14.939853 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":85,"server-negotiator.queue_time_us":41}
I20260806 09:29:14.943122 31026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:14.901872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.61:41373 (local address 127.0.0.1:43068)
0806 09:29:14.916968 (+ 15096us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:14.916991 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:14.917131 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:14.917775 (+ 644us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:14.917780 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:14.918467 (+ 687us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:29:14.918711 (+ 244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:14.918720 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:14.924446 (+ 5726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:14.924453 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:14.926441 (+ 1988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:14.926456 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:14.927685 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:14.927691 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:14.927947 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:14.928624 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:29:14.928645 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:29:14.931252 (+ 2607us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:29:14.938853 (+ 7601us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:29:14.938857 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:29:14.938861 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:29:14.939360 (+ 499us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:29:14.941160 (+ 1800us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:29:14.941166 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:29:14.941171 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:29:14.941399 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:29:14.942140 (+ 741us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:29:14.942145 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:29:14.942375 (+ 230us) client_negotiation.cc:769] Sending connection context
0806 09:29:14.942548 (+ 173us) client_negotiation.cc:240] Negotiation successful
0806 09:29:14.942786 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14916,"mutex_wait_us":56,"thread_start_us":14647,"threads_started":1}
I20260806 09:29:14.944041 30889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:14.901598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43068 (local address 127.20.113.61:41373)
0806 09:29:14.901816 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:14.901824 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:14.917083 (+ 15259us) server_negotiation.cc:407] Connection header received
0806 09:29:14.917375 (+ 292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:14.917381 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:14.917477 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:14.917578 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:14.918879 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:14.919975 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:14.926616 (+ 6641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:14.927256 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:14.927372 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:14.931503 (+ 4131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:29:14.931510 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:29:14.931514 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:29:14.931556 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:29:14.938725 (+ 7169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:14.940825 (+ 2100us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:14.940832 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:14.940837 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:14.941007 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:14.941597 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:14.941602 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:14.941606 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:14.941998 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:29:14.942214 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:14.943488 (+ 1274us) server_negotiation.cc:299] Negotiation successful
0806 09:29:14.943670 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
Aug 06 09:29:14 dist-test-slave-45dx krb5kdc[29852](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008551, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.20.113.1@KRBTEST.COM
Aug 06 09:29:14 dist-test-slave-45dx krb5kdc[29852](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008551, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.113.62@KRBTEST.COM for oryx/127.20.113.60@KRBTEST.COM
I20260806 09:29:14.961416 30751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:14.925893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.60:42483 (local address 127.0.0.1:60256)
0806 09:29:14.926185 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:14.926209 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:14.926345 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:14.933367 (+ 7022us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:14.933374 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:14.933891 (+ 517us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:29:14.934146 (+ 255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:14.934160 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:14.949978 (+ 15818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:14.949984 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:14.950861 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:14.950876 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:14.951887 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:14.951893 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:14.952014 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:14.952579 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:29:14.952597 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:29:14.954876 (+ 2279us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:29:14.958827 (+ 3951us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:29:14.958831 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:29:14.958833 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:29:14.959191 (+ 358us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:29:14.959609 (+ 418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:29:14.959618 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:29:14.959621 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:29:14.959746 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:29:14.960348 (+ 602us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:29:14.960353 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:29:14.960547 (+ 194us) client_negotiation.cc:769] Sending connection context
0806 09:29:14.960709 (+ 162us) client_negotiation.cc:240] Negotiation successful
0806 09:29:14.961011 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142}
I20260806 09:29:14.961447 31029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:14.925951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60256 (local address 127.20.113.60:42483)
0806 09:29:14.932797 (+ 6846us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:14.932808 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:14.932846 (+ 38us) server_negotiation.cc:407] Connection header received
0806 09:29:14.932949 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:14.932955 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:14.933050 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:14.934307 (+ 1257us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:14.948736 (+ 14429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:14.949816 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:14.951021 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:14.951542 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:14.951670 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:14.955101 (+ 3431us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:29:14.955106 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:29:14.955109 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:29:14.955140 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:29:14.958668 (+ 3528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:14.959311 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:14.959317 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:14.959321 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:14.959480 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:14.959859 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:14.959865 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:14.959869 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:14.960183 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:29:14.960364 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:14.960811 (+ 447us) server_negotiation.cc:299] Negotiation successful
0806 09:29:14.961018 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6691,"thread_start_us":6545,"threads_started":1}
I20260806 09:29:14.961953 31030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:14.934601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36234 (local address 127.20.113.1:37689)
0806 09:29:14.935311 (+ 710us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:14.935322 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:14.935353 (+ 31us) server_negotiation.cc:407] Connection header received
0806 09:29:14.935444 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:14.935453 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:14.935741 (+ 288us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:14.935934 (+ 193us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:14.937887 (+ 1953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:14.939809 (+ 1922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:14.942599 (+ 2790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:14.943624 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:14.943775 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:14.950530 (+ 6755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:29:14.950547 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:29:14.950567 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:29:14.950614 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:29:14.957141 (+ 6527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:14.958085 (+ 944us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:14.958096 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:14.958100 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:14.958299 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:14.958787 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:14.958793 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:14.958797 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:14.959329 (+ 532us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:29:14.960835 (+ 1506us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:14.961024 (+ 189us) server_negotiation.cc:299] Negotiation successful
0806 09:29:14.961349 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":529,"thread_start_us":351,"threads_started":1}
I20260806 09:29:14.963287 29915 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.20.113.60:42483: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "e8ec8de97a86401e948b6ffb6a72e7d1"
member_type: NON_VOTER
last_known_addr {
host: "127.20.113.60"
port: 42483
}
attrs {
promote: true
}
}
dest_uuid: "0cbb5d09bd23462caa29c64dd33fdd25"
cas_config_opid_index: 5
I20260806 09:29:14.963605 30184 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:29:14.964428 29915 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0cbb5d09bd23462caa29c64dd33fdd25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0cbb5d09bd23462caa29c64dd33fdd25" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 45173 } } peers { permanent_uuid: "377f07a0afaa4800aa98d87714775150" member_type: NON_VOTER last_known_addr { host: "127.20.113.61" port: 41373 } attrs { promote: true } } peers { permanent_uuid: "e8ec8de97a86401e948b6ffb6a72e7d1" member_type: NON_VOTER last_known_addr { host: "127.20.113.60" port: 42483 } attrs { promote: true } }
I20260806 09:29:14.966472 30757 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0cbb5d09bd23462caa29c64dd33fdd25 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER e8ec8de97a86401e948b6ffb6a72e7d1 (127.20.113.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0cbb5d09bd23462caa29c64dd33fdd25" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 45173 } } peers { permanent_uuid: "377f07a0afaa4800aa98d87714775150" member_type: NON_VOTER last_known_addr { host: "127.20.113.61" port: 41373 } attrs { promote: true } } peers { permanent_uuid: "e8ec8de97a86401e948b6ffb6a72e7d1" member_type: NON_VOTER last_known_addr { host: "127.20.113.60" port: 42483 } attrs { promote: true } } }
I20260806 09:29:14.967073 30953 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e8ec8de97a86401e948b6ffb6a72e7d1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:29:14.967672 30827 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 377f07a0afaa4800aa98d87714775150 [term 1 LEARNER]: Refusing update from remote peer 0cbb5d09bd23462caa29c64dd33fdd25: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260806 09:29:14.972115 30953 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e8ec8de97a86401e948b6ffb6a72e7d1 [term 1 FOLLOWER]: Refusing update from remote peer 0cbb5d09bd23462caa29c64dd33fdd25: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260806 09:29:14.974017 31033 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0cbb5d09bd23462caa29c64dd33fdd25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "377f07a0afaa4800aa98d87714775150" member_type: NON_VOTER last_known_addr { host: "127.20.113.61" port: 41373 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260806 09:29:14.980155 30757 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.20.113.60:42483
I20260806 09:29:14.981112 30761 master.cc:537] Master@127.20.113.60:42483 shutting down...
I20260806 09:29:14.981094 31035 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0cbb5d09bd23462caa29c64dd33fdd25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8ec8de97a86401e948b6ffb6a72e7d1" member_type: NON_VOTER last_known_addr { host: "127.20.113.60" port: 42483 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 09:29:14.993570 30827 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 377f07a0afaa4800aa98d87714775150 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER e8ec8de97a86401e948b6ffb6a72e7d1 (127.20.113.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0cbb5d09bd23462caa29c64dd33fdd25" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 45173 } } peers { permanent_uuid: "377f07a0afaa4800aa98d87714775150" member_type: NON_VOTER last_known_addr { host: "127.20.113.61" port: 41373 } attrs { promote: true } } peers { permanent_uuid: "e8ec8de97a86401e948b6ffb6a72e7d1" member_type: NON_VOTER last_known_addr { host: "127.20.113.60" port: 42483 } attrs { promote: true } } }
I20260806 09:29:14.995338 30761 raft_consensus.cc:2229] T 00000000000000000000000000000000 P e8ec8de97a86401e948b6ffb6a72e7d1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 09:29:14.995460 31036 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0cbb5d09bd23462caa29c64dd33fdd25 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "0cbb5d09bd23462caa29c64dd33fdd25" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0cbb5d09bd23462caa29c64dd33fdd25" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 45173 } } peers { permanent_uuid: "377f07a0afaa4800aa98d87714775150" member_type: NON_VOTER last_known_addr { host: "127.20.113.61" port: 41373 } attrs { promote: true } } peers { permanent_uuid: "e8ec8de97a86401e948b6ffb6a72e7d1" member_type: NON_VOTER last_known_addr { host: "127.20.113.60" port: 42483 } attrs { promote: true } } }
I20260806 09:29:14.995769 30761 raft_consensus.cc:2258] T 00000000000000000000000000000000 P e8ec8de97a86401e948b6ffb6a72e7d1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 09:29:14.995882 31036 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0cbb5d09bd23462caa29c64dd33fdd25 [sys.catalog]: This master's current role is: LEADER
I20260806 09:29:14.995985 30761 tablet_replica.cc:330] T 00000000000000000000000000000000 P e8ec8de97a86401e948b6ffb6a72e7d1: stopping tablet replica
I20260806 09:29:15.000089 30873 sys_catalog.cc:455] T 00000000000000000000000000000000 P 377f07a0afaa4800aa98d87714775150 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0cbb5d09bd23462caa29c64dd33fdd25" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0cbb5d09bd23462caa29c64dd33fdd25" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 45173 } } peers { permanent_uuid: "377f07a0afaa4800aa98d87714775150" member_type: NON_VOTER last_known_addr { host: "127.20.113.61" port: 41373 } attrs { promote: true } } peers { permanent_uuid: "e8ec8de97a86401e948b6ffb6a72e7d1" member_type: NON_VOTER last_known_addr { host: "127.20.113.60" port: 42483 } attrs { promote: true } } }
I20260806 09:29:15.000468 30873 sys_catalog.cc:458] T 00000000000000000000000000000000 P 377f07a0afaa4800aa98d87714775150 [sys.catalog]: This master's current role is: LEARNER
I20260806 09:29:15.004781 30761 master.cc:552] Master@127.20.113.60:42483 shutdown complete.
I20260806 09:29:15.004868 30761 master_runner.cc:305] Clearing existing system tablet
I20260806 09:29:15.005297 31036 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 0cbb5d09bd23462caa29c64dd33fdd25: attempting to promote NON_VOTER 377f07a0afaa4800aa98d87714775150 to VOTER
I20260806 09:29:15.005324 30761 fs_manager.cc:359] Metadata directory not provided
I20260806 09:29:15.005415 30761 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:29:15.006413 31036 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0cbb5d09bd23462caa29c64dd33fdd25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0cbb5d09bd23462caa29c64dd33fdd25" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 45173 } } peers { permanent_uuid: "377f07a0afaa4800aa98d87714775150" member_type: VOTER last_known_addr { host: "127.20.113.61" port: 41373 } attrs { promote: false } } peers { permanent_uuid: "e8ec8de97a86401e948b6ffb6a72e7d1" member_type: NON_VOTER last_known_addr { host: "127.20.113.60" port: 42483 } attrs { promote: true } }
I20260806 09:29:15.009765 30751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:15.009170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.60:42483 (local address 127.0.0.1:60270)
0806 09:29:15.009381 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:15.009545 (+ 164us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.60:42483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260806 09:29:15.010921 30827 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 377f07a0afaa4800aa98d87714775150 [term 1 LEARNER]: Refusing update from remote peer 0cbb5d09bd23462caa29c64dd33fdd25: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260806 09:29:15.011524 30761 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.002s sys 0.004s
I20260806 09:29:15.012194 30757 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0cbb5d09bd23462caa29c64dd33fdd25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "377f07a0afaa4800aa98d87714775150" member_type: VOTER last_known_addr { host: "127.20.113.61" port: 41373 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260806 09:29:15.013592 30751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:15.013007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.60:42483 (local address 127.0.0.1:60276)
0806 09:29:15.013208 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:15.013361 (+ 153us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.60:42483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260806 09:29:15.015352 29895 consensus_peers.cc:487] T 00000000000000000000000000000000 P 0cbb5d09bd23462caa29c64dd33fdd25 -> Peer e8ec8de97a86401e948b6ffb6a72e7d1 (127.20.113.60:42483): Couldn't send request to peer e8ec8de97a86401e948b6ffb6a72e7d1. Status: Network error: Client connection negotiation failed: client connection to 127.20.113.60:42483: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260806 09:29:15.016398 31037 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0cbb5d09bd23462caa29c64dd33fdd25 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 377f07a0afaa4800aa98d87714775150 (127.20.113.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0cbb5d09bd23462caa29c64dd33fdd25" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 45173 } } peers { permanent_uuid: "377f07a0afaa4800aa98d87714775150" member_type: VOTER last_known_addr { host: "127.20.113.61" port: 41373 } attrs { promote: false } } peers { permanent_uuid: "e8ec8de97a86401e948b6ffb6a72e7d1" member_type: NON_VOTER last_known_addr { host: "127.20.113.60" port: 42483 } attrs { promote: true } } }
I20260806 09:29:15.021454 31049 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:29:15.021848 31035 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0cbb5d09bd23462caa29c64dd33fdd25 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "0cbb5d09bd23462caa29c64dd33fdd25" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0cbb5d09bd23462caa29c64dd33fdd25" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 45173 } } peers { permanent_uuid: "377f07a0afaa4800aa98d87714775150" member_type: VOTER last_known_addr { host: "127.20.113.61" port: 41373 } attrs { promote: false } } peers { permanent_uuid: "e8ec8de97a86401e948b6ffb6a72e7d1" member_type: NON_VOTER last_known_addr { host: "127.20.113.60" port: 42483 } attrs { promote: true } } }
I20260806 09:29:15.022222 31035 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0cbb5d09bd23462caa29c64dd33fdd25 [sys.catalog]: This master's current role is: LEADER
I20260806 09:29:15.022197 31036 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0cbb5d09bd23462caa29c64dd33fdd25 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0cbb5d09bd23462caa29c64dd33fdd25" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0cbb5d09bd23462caa29c64dd33fdd25" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 45173 } } peers { permanent_uuid: "377f07a0afaa4800aa98d87714775150" member_type: VOTER last_known_addr { host: "127.20.113.61" port: 41373 } attrs { promote: false } } peers { permanent_uuid: "e8ec8de97a86401e948b6ffb6a72e7d1" member_type: NON_VOTER last_known_addr { host: "127.20.113.60" port: 42483 } attrs { promote: true } } }
I20260806 09:29:15.022450 31036 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0cbb5d09bd23462caa29c64dd33fdd25 [sys.catalog]: This master's current role is: LEADER
I20260806 09:29:15.022532 30761 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:29:15.022869 30761 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.005s sys 0.004s
I20260806 09:29:15.023005 30761 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/master-2/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/master-2/wal
uuid: "e8ec8de97a86401e948b6ffb6a72e7d1"
format_stamp: "Formatted at 2026-08-06 09:29:14 on dist-test-slave-45dx"
server_key: "349707703198dbc3a5c4780f075bd1b7"
server_key_iv: "b85360db13c32a6d7b34c4723b64ad6b"
server_key_version: "encryptionkey@0"
I20260806 09:29:15.023283 30827 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 377f07a0afaa4800aa98d87714775150 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 377f07a0afaa4800aa98d87714775150 (127.20.113.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0cbb5d09bd23462caa29c64dd33fdd25" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 45173 } } peers { permanent_uuid: "377f07a0afaa4800aa98d87714775150" member_type: VOTER last_known_addr { host: "127.20.113.61" port: 41373 } attrs { promote: false } } peers { permanent_uuid: "e8ec8de97a86401e948b6ffb6a72e7d1" member_type: NON_VOTER last_known_addr { host: "127.20.113.60" port: 42483 } attrs { promote: true } } }
I20260806 09:29:15.027498 30873 sys_catalog.cc:455] T 00000000000000000000000000000000 P 377f07a0afaa4800aa98d87714775150 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0cbb5d09bd23462caa29c64dd33fdd25" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0cbb5d09bd23462caa29c64dd33fdd25" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 45173 } } peers { permanent_uuid: "377f07a0afaa4800aa98d87714775150" member_type: VOTER last_known_addr { host: "127.20.113.61" port: 41373 } attrs { promote: false } } peers { permanent_uuid: "e8ec8de97a86401e948b6ffb6a72e7d1" member_type: NON_VOTER last_known_addr { host: "127.20.113.60" port: 42483 } attrs { promote: true } } }
I20260806 09:29:15.027740 30873 sys_catalog.cc:458] T 00000000000000000000000000000000 P 377f07a0afaa4800aa98d87714775150 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 09:29:15.028265 30761 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P e8ec8de97a86401e948b6ffb6a72e7d1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 09:29:15.033823 30761 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P e8ec8de97a86401e948b6ffb6a72e7d1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260806 09:29:15.034024 30761 log.cc:1196] T 00000000000000000000000000000000 P e8ec8de97a86401e948b6ffb6a72e7d1: Deleting WAL directory at /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260806 09:29:15.034572 30761 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P e8ec8de97a86401e948b6ffb6a72e7d1: Deleting consensus metadata
I20260806 09:29:15.034879 30761 master_runner.cc:315] Copying system tablet from 127.20.113.62:45173
I20260806 09:29:15.037948 30761 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P e8ec8de97a86401e948b6ffb6a72e7d1: tablet copy: Beginning tablet copy session from remote peer at address 127.20.113.62:45173
I20260806 09:29:15.051857 30994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:15.038792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59206 (local address 127.20.113.62:45173)
0806 09:29:15.039190 (+ 398us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:15.039196 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:15.039518 (+ 322us) server_negotiation.cc:407] Connection header received
0806 09:29:15.039767 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:15.039773 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:15.039858 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:15.040001 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:15.041131 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:15.042092 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:15.043552 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:15.044030 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:15.044171 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:15.046085 (+ 1914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:29:15.046092 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:29:15.046096 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:29:15.046139 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:29:15.049107 (+ 2968us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:15.049780 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:15.049790 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:15.049792 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:15.049922 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:15.050336 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:15.050342 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:15.050345 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:15.050654 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:29:15.050830 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:15.051241 (+ 411us) server_negotiation.cc:299] Negotiation successful
0806 09:29:15.051462 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":237}
I20260806 09:29:15.051878 31056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:15.038787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:45173 (local address 127.0.0.1:59206)
0806 09:29:15.039384 (+ 597us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:15.039409 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:15.039595 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:15.040032 (+ 437us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:15.040037 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:15.040644 (+ 607us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:29:15.040909 (+ 265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:15.040923 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:15.042340 (+ 1417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:15.042343 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:15.043404 (+ 1061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:15.043417 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:15.044206 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:15.044212 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:15.044410 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:15.045100 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:29:15.045118 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:29:15.045859 (+ 741us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:29:15.049260 (+ 3401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:29:15.049266 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:29:15.049270 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:29:15.049647 (+ 377us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:29:15.050041 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:29:15.050046 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:29:15.050050 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:29:15.050221 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:29:15.050794 (+ 573us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:29:15.050799 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:29:15.050978 (+ 179us) client_negotiation.cc:769] Sending connection context
0806 09:29:15.051129 (+ 151us) client_negotiation.cc:240] Negotiation successful
0806 09:29:15.051451 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":409,"threads_started":1}
I20260806 09:29:15.052863 29950 tablet_copy_service.cc:140] P 0cbb5d09bd23462caa29c64dd33fdd25: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer e8ec8de97a86401e948b6ffb6a72e7d1 ({username='oryx', principal='oryx/127.20.113.60@KRBTEST.COM'} at 127.0.0.1:59206)
I20260806 09:29:15.053054 29950 tablet_copy_service.cc:161] P 0cbb5d09bd23462caa29c64dd33fdd25: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer e8ec8de97a86401e948b6ffb6a72e7d1 at {username='oryx', principal='oryx/127.20.113.60@KRBTEST.COM'} at 127.0.0.1:59206: session id = e8ec8de97a86401e948b6ffb6a72e7d1-00000000000000000000000000000000
I20260806 09:29:15.055421 29950 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 0cbb5d09bd23462caa29c64dd33fdd25: Tablet Copy: opened 0 blocks and 1 log segments
I20260806 09:29:15.057075 30761 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260806 09:29:15.061687 30761 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P e8ec8de97a86401e948b6ffb6a72e7d1: tablet copy: Starting download of 0 data blocks...
I20260806 09:29:15.062002 30761 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P e8ec8de97a86401e948b6ffb6a72e7d1: tablet copy: Starting download of 1 WAL segments...
I20260806 09:29:15.064266 30761 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P e8ec8de97a86401e948b6ffb6a72e7d1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260806 09:29:15.067802 29950 tablet_copy_service.cc:342] P 0cbb5d09bd23462caa29c64dd33fdd25: Request end of tablet copy session e8ec8de97a86401e948b6ffb6a72e7d1-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.20.113.60@KRBTEST.COM'} at 127.0.0.1:59206
I20260806 09:29:15.067945 29950 tablet_copy_service.cc:433] P 0cbb5d09bd23462caa29c64dd33fdd25: ending tablet copy session e8ec8de97a86401e948b6ffb6a72e7d1-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer e8ec8de97a86401e948b6ffb6a72e7d1
I20260806 09:29:15.078320 30761 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:29:15.080933 31057 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:29:15.085914 31058 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:29:15.091158 30761 server_base.cc:1034] running on GCE node
W20260806 09:29:15.091709 31061 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:29:15.092228 30761 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:29:15.092973 30761 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:29:15.094177 30761 hybrid_clock.cc:648] HybridClock initialized: now 1786008555094122 us; error 57 us; skew 500 ppm
I20260806 09:29:15.094426 30761 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:29:15 dist-test-slave-45dx krb5kdc[29852](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008555, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.113.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:29:15.096403 30761 init.cc:377] Logged in from keytab as oryx/127.20.113.60@KRBTEST.COM (short username oryx)
I20260806 09:29:15.097743 30761 webserver.cc:460] Webserver started at http://127.20.113.60:43385/ using document root <none> and password file <none>
I20260806 09:29:15.098079 30761 fs_manager.cc:359] Metadata directory not provided
I20260806 09:29:15.098219 30761 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:29:15.101141 30761 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260806 09:29:15.103061 31072 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:29:15.103534 30761 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20260806 09:29:15.103668 30761 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/master-2/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/master-2/wal
uuid: "e8ec8de97a86401e948b6ffb6a72e7d1"
format_stamp: "Formatted at 2026-08-06 09:29:14 on dist-test-slave-45dx"
server_key: "349707703198dbc3a5c4780f075bd1b7"
server_key_iv: "b85360db13c32a6d7b34c4723b64ad6b"
server_key_version: "encryptionkey@0"
I20260806 09:29:15.103878 30761 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:29:15.133533 30761 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:29:15.134482 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 30010
I20260806 09:29:15.137287 30761 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:29:15.170723 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.113.1:37689
--local_ip_for_outbound_sockets=127.20.113.1
--tserver_master_addrs=127.20.113.62:45173
--webserver_port=41597
--webserver_interface=127.20.113.1
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/krb5kdc/oryx_127.20.113.1.keytab
--principal=oryx/127.20.113.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:41701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.20.113.62:45173,127.20.113.61:41373
--tserver_master_addrs=127.20.113.62:45173,127.20.113.61:41373,127.20.113.60:42483 with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 09:29:15.180802 30761 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.60:42483
I20260806 09:29:15.180877 31148 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.60:42483 every 8 connection(s)
I20260806 09:29:15.182065 30761 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/master-2/data/info.pb
I20260806 09:29:15.187412 31151 sys_catalog.cc:263] Verifying existing consensus state
I20260806 09:29:15.191426 31151 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8ec8de97a86401e948b6ffb6a72e7d1: Bootstrap starting.
I20260806 09:29:15.216357 31151 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8ec8de97a86401e948b6ffb6a72e7d1: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 09:29:15.216910 31151 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8ec8de97a86401e948b6ffb6a72e7d1: Bootstrap complete.
I20260806 09:29:15.218032 31151 raft_consensus.cc:348] T 00000000000000000000000000000000 P e8ec8de97a86401e948b6ffb6a72e7d1 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0cbb5d09bd23462caa29c64dd33fdd25" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 45173 } } peers { permanent_uuid: "377f07a0afaa4800aa98d87714775150" member_type: VOTER last_known_addr { host: "127.20.113.61" port: 41373 } attrs { promote: false } } peers { permanent_uuid: "e8ec8de97a86401e948b6ffb6a72e7d1" member_type: NON_VOTER last_known_addr { host: "127.20.113.60" port: 42483 } attrs { promote: true } }
I20260806 09:29:15.218340 31151 raft_consensus.cc:729] T 00000000000000000000000000000000 P e8ec8de97a86401e948b6ffb6a72e7d1 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e8ec8de97a86401e948b6ffb6a72e7d1, State: Initialized, Role: LEARNER
I20260806 09:29:15.218606 31151 consensus_queue.cc:260] T 00000000000000000000000000000000 P e8ec8de97a86401e948b6ffb6a72e7d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0cbb5d09bd23462caa29c64dd33fdd25" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 45173 } } peers { permanent_uuid: "377f07a0afaa4800aa98d87714775150" member_type: VOTER last_known_addr { host: "127.20.113.61" port: 41373 } attrs { promote: false } } peers { permanent_uuid: "e8ec8de97a86401e948b6ffb6a72e7d1" member_type: NON_VOTER last_known_addr { host: "127.20.113.60" port: 42483 } attrs { promote: true } }
I20260806 09:29:15.220803 31156 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8ec8de97a86401e948b6ffb6a72e7d1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0cbb5d09bd23462caa29c64dd33fdd25" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 45173 } } peers { permanent_uuid: "377f07a0afaa4800aa98d87714775150" member_type: VOTER last_known_addr { host: "127.20.113.61" port: 41373 } attrs { promote: false } } peers { permanent_uuid: "e8ec8de97a86401e948b6ffb6a72e7d1" member_type: NON_VOTER last_known_addr { host: "127.20.113.60" port: 42483 } attrs { promote: true } } }
I20260806 09:29:15.221143 31151 sys_catalog.cc:564] T 00000000000000000000000000000000 P e8ec8de97a86401e948b6ffb6a72e7d1 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 09:29:15.221379 31156 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8ec8de97a86401e948b6ffb6a72e7d1 [sys.catalog]: This master's current role is: LEARNER
I20260806 09:29:15.243237 31169 catalog_manager.cc:1259] Loaded cluster ID: e50f8b022b59446eaf1c3aaefb8eb3e7
I20260806 09:29:15.243359 31169 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e8ec8de97a86401e948b6ffb6a72e7d1: loading cluster ID for follower catalog manager: success
I20260806 09:29:15.246767 31169 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e8ec8de97a86401e948b6ffb6a72e7d1: acquiring CA information for follower catalog manager: success
I20260806 09:29:15.249115 31169 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e8ec8de97a86401e948b6ffb6a72e7d1: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260806 09:29:15.545759 31154 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:29:15.546027 31154 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:29:15.546125 31154 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:29:15.553653 31154 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:29:15.553761 31154 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:29:15.553953 31154 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.113.1
I20260806 09:29:15.563628 31154 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:41701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/krb5kdc/oryx_127.20.113.1.keytab
--principal=oryx/127.20.113.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.113.1:37689
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.113.1
--webserver_port=41597
--tserver_master_addrs=127.20.113.62:45173,127.20.113.61:41373,127.20.113.60:42483
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.113.1
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:29:15.565101 31154 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:29:15.566648 31154 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:29:15.581676 31180 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:29:15.581475 31177 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:29:15.582206 31176 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:29:15.582892 31181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:15.575178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.60:42483 (local address 127.0.0.1:60286)
0806 09:29:15.575693 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:15.575718 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:15.575880 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:15.576344 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:15.576349 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:15.576382 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:29:15.576588 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:15.576600 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:15.577907 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:15.577914 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:15.579247 (+ 1333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:15.579267 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:15.581586 (+ 2319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:15.581595 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:15.581944 (+ 349us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:15.582009 (+ 65us) client_negotiation.cc:769] Sending connection context
0806 09:29:15.582118 (+ 109us) client_negotiation.cc:240] Negotiation successful
0806 09:29:15.582359 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":225,"threads_started":1}
I20260806 09:29:15.584545 31182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:15.575363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60286 (local address 127.20.113.60:42483)
0806 09:29:15.575954 (+ 591us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:15.575964 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:15.575985 (+ 21us) server_negotiation.cc:407] Connection header received
0806 09:29:15.576068 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:15.576071 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:15.576146 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:15.576283 (+ 137us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:29:15.576767 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:15.577734 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:15.579489 (+ 1755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:15.581244 (+ 1755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:15.583402 (+ 2158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:15.583730 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:15.583806 (+ 76us) server_negotiation.cc:299] Negotiation successful
0806 09:29:15.584042 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":430,"thread_start_us":253,"threads_started":1}
I20260806 09:29:15.585521 31107 raft_consensus.cc:1203] T 00000000000000000000000000000000 P e8ec8de97a86401e948b6ffb6a72e7d1 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260806 09:29:15.586184 31154 server_base.cc:1034] running on GCE node
I20260806 09:29:15.586246 31156 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8ec8de97a86401e948b6ffb6a72e7d1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0cbb5d09bd23462caa29c64dd33fdd25. Latest consensus state: current_term: 1 leader_uuid: "0cbb5d09bd23462caa29c64dd33fdd25" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0cbb5d09bd23462caa29c64dd33fdd25" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 45173 } } peers { permanent_uuid: "377f07a0afaa4800aa98d87714775150" member_type: VOTER last_known_addr { host: "127.20.113.61" port: 41373 } attrs { promote: false } } peers { permanent_uuid: "e8ec8de97a86401e948b6ffb6a72e7d1" member_type: NON_VOTER last_known_addr { host: "127.20.113.60" port: 42483 } attrs { promote: true } } }
I20260806 09:29:15.586571 31156 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8ec8de97a86401e948b6ffb6a72e7d1 [sys.catalog]: This master's current role is: LEARNER
I20260806 09:29:15.587051 31154 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:29:15.588169 31154 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:29:15.589426 31154 hybrid_clock.cc:648] HybridClock initialized: now 1786008555589369 us; error 71 us; skew 500 ppm
I20260806 09:29:15.589788 31154 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:29:15 dist-test-slave-45dx krb5kdc[29852](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008555, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.113.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:29:15.593243 31154 init.cc:377] Logged in from keytab as oryx/127.20.113.1@KRBTEST.COM (short username oryx)
I20260806 09:29:15.594940 31154 webserver.cc:460] Webserver started at http://127.20.113.1:41597/ using document root <none> and password file <none>
I20260806 09:29:15.595711 31154 fs_manager.cc:359] Metadata directory not provided
I20260806 09:29:15.595890 31154 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:29:15.601488 31154 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 09:29:15.604525 31192 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:29:15.606053 31154 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 09:29:15.606236 31154 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/ts-0/wal
uuid: "f151cc51113947f6a0ea2f463ed20681"
format_stamp: "Formatted at 2026-08-06 09:29:12 on dist-test-slave-45dx"
server_key: "cd9e3d7a793e12dff6531e4b8724e1d1"
server_key_iv: "3ffa1a02aabe475403069673efde3baf"
server_key_version: "encryptionkey@0"
I20260806 09:29:15.607239 31154 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:29:15.624792 31154 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:29:15.629276 31154 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:29:15.629603 31154 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:29:15.630681 31154 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:29:15.632119 31154 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:29:15.632229 31154 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:29:15.632416 31154 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:29:15.632519 31154 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:29:15.671875 31154 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.1:37689
I20260806 09:29:15.671957 31368 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.1:37689 every 8 connection(s)
I20260806 09:29:15.673427 31154 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/ts-0/data/info.pb
I20260806 09:29:15.676682 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 31154
I20260806 09:29:15.677093 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 30200
Aug 06 09:29:15 dist-test-slave-45dx krb5kdc[29852](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008555, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.113.1@KRBTEST.COM for oryx/127.20.113.60@KRBTEST.COM
Aug 06 09:29:15 dist-test-slave-45dx krb5kdc[29852](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008551, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.20.113.2@KRBTEST.COM
Aug 06 09:29:15 dist-test-slave-45dx krb5kdc[29852](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008555, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.113.1@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM
Aug 06 09:29:15 dist-test-slave-45dx krb5kdc[29852](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008555, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.113.1@KRBTEST.COM for oryx/127.20.113.61@KRBTEST.COM
I20260806 09:29:15.704931 31182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:15.676626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.1:46145 (local address 127.20.113.60:42483)
0806 09:29:15.676909 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:15.676919 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:15.678526 (+ 1607us) server_negotiation.cc:407] Connection header received
0806 09:29:15.681683 (+ 3157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:15.681691 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:15.681817 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:15.681901 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:15.686651 (+ 4750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:15.687857 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:15.690716 (+ 2859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:15.691381 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:15.691495 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:15.696359 (+ 4864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:29:15.696367 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:29:15.696372 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:29:15.696417 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:29:15.700179 (+ 3762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:15.701537 (+ 1358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:15.701543 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:15.701547 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:15.701729 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:15.702404 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:15.702410 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:15.702414 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:15.702814 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:29:15.702985 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:15.704161 (+ 1176us) server_negotiation.cc:299] Negotiation successful
0806 09:29:15.704449 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260806 09:29:15.707361 31379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:15.677339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.60:42483 (local address 127.20.113.1:46145)
0806 09:29:15.678360 (+ 1021us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:15.678411 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:15.681296 (+ 2885us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:15.682528 (+ 1232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:15.682543 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:15.683470 (+ 927us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:29:15.684397 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:15.684426 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:15.688076 (+ 3650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:15.688085 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:15.690530 (+ 2445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:15.690544 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:15.691712 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:15.691721 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:15.692028 (+ 307us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:15.692873 (+ 845us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:29:15.692901 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:29:15.696079 (+ 3178us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:29:15.700418 (+ 4339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:29:15.700426 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:29:15.700459 (+ 33us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:29:15.701364 (+ 905us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:29:15.701941 (+ 577us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:29:15.701946 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:29:15.701950 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:29:15.702251 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:29:15.703089 (+ 838us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:29:15.703100 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:29:15.703656 (+ 556us) client_negotiation.cc:769] Sending connection context
0806 09:29:15.705155 (+ 1499us) client_negotiation.cc:240] Negotiation successful
0806 09:29:15.705757 (+ 602us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":665,"mutex_wait_us":31,"thread_start_us":880,"threads_started":1}
I20260806 09:29:15.708856 31385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:15.678808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44832 (local address 127.20.113.2:38959)
0806 09:29:15.685654 (+ 6846us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:15.685665 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:15.685699 (+ 34us) server_negotiation.cc:407] Connection header received
0806 09:29:15.685780 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:15.685791 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:15.686061 (+ 270us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:15.686317 (+ 256us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:15.688344 (+ 2027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:15.689995 (+ 1651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:15.691912 (+ 1917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:15.693137 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:15.698100 (+ 4963us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:15.699883 (+ 1783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:29:15.699919 (+ 36us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:29:15.699937 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:29:15.699982 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:29:15.704185 (+ 4203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:15.705442 (+ 1257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:15.705452 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:15.705455 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:15.705623 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:15.706103 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:15.706109 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:15.706111 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:15.706560 (+ 449us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:29:15.706851 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:15.708140 (+ 1289us) server_negotiation.cc:299] Negotiation successful
0806 09:29:15.708448 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6084,"thread_start_us":5909,"threads_started":1}
I20260806 09:29:15.709223 31369 heartbeater.cc:347] Connected to a master server at 127.20.113.60:42483
I20260806 09:29:15.709575 31369 heartbeater.cc:464] Registering TS with master...
I20260806 09:29:15.710052 31386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:15.679161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.1:40055 (local address 127.20.113.62:45173)
0806 09:29:15.685542 (+ 6381us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:15.685551 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:15.689909 (+ 4358us) server_negotiation.cc:407] Connection header received
0806 09:29:15.689987 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:15.689992 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:15.690067 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:15.690177 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:15.692114 (+ 1937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:15.693307 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:15.695649 (+ 2342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:15.696389 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:15.696522 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:15.701614 (+ 5092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:29:15.701621 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:29:15.701625 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:29:15.701671 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:29:15.706223 (+ 4552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:15.707165 (+ 942us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:15.707172 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:15.707177 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:15.707329 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:15.707765 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:15.707771 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:15.707775 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:15.708115 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:29:15.708261 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:15.709385 (+ 1124us) server_negotiation.cc:299] Negotiation successful
0806 09:29:15.709662 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6222,"thread_start_us":6097,"threads_started":1}
I20260806 09:29:15.710163 30382 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:29:15.710536 31369 heartbeater.cc:511] Master 127.20.113.60:42483 requested a full tablet report, sending...
I20260806 09:29:15.711555 31388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:15.680035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.1:53267 (local address 127.20.113.61:41373)
0806 09:29:15.688750 (+ 8715us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:15.688756 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:15.690147 (+ 1391us) server_negotiation.cc:407] Connection header received
0806 09:29:15.690220 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:15.690226 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:15.690322 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:15.690423 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:15.691961 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:15.693063 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:15.695799 (+ 2736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:15.696480 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:15.696580 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:15.702789 (+ 6209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:29:15.702796 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:29:15.702800 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:29:15.702841 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:29:15.706783 (+ 3942us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:15.707943 (+ 1160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:15.707952 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:15.707954 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:15.708107 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:15.708702 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:15.708708 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:15.708710 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:15.709462 (+ 752us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:29:15.709595 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:15.710701 (+ 1106us) server_negotiation.cc:299] Negotiation successful
0806 09:29:15.710971 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8594,"thread_start_us":8451,"threads_started":1}
I20260806 09:29:15.712774 31384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:15.679274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.61:41373 (local address 127.20.113.1:53267)
0806 09:29:15.689906 (+ 10632us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:15.689925 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:15.690042 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:15.690630 (+ 588us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:15.690635 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:15.691273 (+ 638us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:29:15.691783 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:15.691798 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:15.693254 (+ 1456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:15.693261 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:15.695627 (+ 2366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:15.695642 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:15.696914 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:15.696923 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:15.697139 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:15.697757 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:29:15.697778 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:29:15.702563 (+ 4785us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:29:15.706952 (+ 4389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:29:15.706962 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:29:15.706965 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:29:15.707792 (+ 827us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:29:15.708354 (+ 562us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:29:15.708358 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:29:15.708361 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:29:15.708571 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:29:15.709751 (+ 1180us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:29:15.709755 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:29:15.709892 (+ 137us) client_negotiation.cc:769] Sending connection context
0806 09:29:15.710039 (+ 147us) client_negotiation.cc:240] Negotiation successful
0806 09:29:15.712361 (+ 2322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10497,"mutex_wait_us":84,"spinlock_wait_cycles":1280,"thread_start_us":9674,"threads_started":1}
I20260806 09:29:15.713285 31094 ts_manager.cc:194] Registered new tserver with Master: f151cc51113947f6a0ea2f463ed20681 (127.20.113.1:37689)
I20260806 09:29:15.713914 31373 heartbeater.cc:347] Connected to a master server at 127.20.113.61:41373
I20260806 09:29:15.714048 31373 heartbeater.cc:464] Registering TS with master...
I20260806 09:29:15.714283 31387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:15.678130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:45173 (local address 127.20.113.1:40055)
0806 09:29:15.689276 (+ 11146us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:15.689300 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:15.689422 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:15.690412 (+ 990us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:15.690418 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:15.691112 (+ 694us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:29:15.691953 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:15.691965 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:15.693482 (+ 1517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:15.693487 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:15.695441 (+ 1954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:15.695459 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:15.696806 (+ 1347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:15.696812 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:15.697009 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:15.697611 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:29:15.697638 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:29:15.701296 (+ 3658us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:29:15.706414 (+ 5118us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:29:15.706418 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:29:15.706422 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:29:15.707020 (+ 598us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:29:15.707474 (+ 454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:29:15.707478 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:29:15.707481 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:29:15.707632 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:29:15.708355 (+ 723us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:29:15.708362 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:29:15.708620 (+ 258us) client_negotiation.cc:769] Sending connection context
0806 09:29:15.708762 (+ 142us) client_negotiation.cc:240] Negotiation successful
0806 09:29:15.709053 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11012,"mutex_wait_us":435,"spinlock_wait_cycles":27392,"thread_start_us":11497,"threads_started":1}
I20260806 09:29:15.714591 31373 heartbeater.cc:511] Master 127.20.113.61:41373 requested a full tablet report, sending...
I20260806 09:29:15.715528 31372 heartbeater.cc:347] Connected to a master server at 127.20.113.62:45173
I20260806 09:29:15.715665 31372 heartbeater.cc:464] Registering TS with master...
I20260806 09:29:15.716104 31372 heartbeater.cc:511] Master 127.20.113.62:45173 requested a full tablet report, sending...
I20260806 09:29:15.716333 30815 ts_manager.cc:194] Registered new tserver with Master: f151cc51113947f6a0ea2f463ed20681 (127.20.113.1:37689)
I20260806 09:29:15.717272 29915 ts_manager.cc:194] Re-registered known tserver with Master: f151cc51113947f6a0ea2f463ed20681 (127.20.113.1:37689)
I20260806 09:29:15.718482 29915 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.20.113.1@KRBTEST.COM'} at 127.20.113.1:40055
I20260806 09:29:15.795367 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 30200
I20260806 09:29:15.820144 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.113.2:38959
--local_ip_for_outbound_sockets=127.20.113.2
--tserver_master_addrs=127.20.113.62:45173
--webserver_port=46583
--webserver_interface=127.20.113.2
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/krb5kdc/oryx_127.20.113.2.keytab
--principal=oryx/127.20.113.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:41701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.20.113.62:45173,127.20.113.61:41373
--tserver_master_addrs=127.20.113.62:45173,127.20.113.61:41373,127.20.113.60:42483 with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 09:29:16.095777 31170 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0cbb5d09bd23462caa29c64dd33fdd25 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0cbb5d09bd23462caa29c64dd33fdd25" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0cbb5d09bd23462caa29c64dd33fdd25" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 45173 } } peers { permanent_uuid: "377f07a0afaa4800aa98d87714775150" member_type: VOTER last_known_addr { host: "127.20.113.61" port: 41373 } attrs { promote: false } } peers { permanent_uuid: "e8ec8de97a86401e948b6ffb6a72e7d1" member_type: NON_VOTER last_known_addr { host: "127.20.113.60" port: 42483 } attrs { promote: true } } }
I20260806 09:29:16.096207 31170 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0cbb5d09bd23462caa29c64dd33fdd25 [sys.catalog]: This master's current role is: LEADER
I20260806 09:29:16.097637 31170 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 0cbb5d09bd23462caa29c64dd33fdd25: attempting to promote NON_VOTER e8ec8de97a86401e948b6ffb6a72e7d1 to VOTER
I20260806 09:29:16.098265 31170 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0cbb5d09bd23462caa29c64dd33fdd25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "0cbb5d09bd23462caa29c64dd33fdd25" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 45173 } } peers { permanent_uuid: "377f07a0afaa4800aa98d87714775150" member_type: VOTER last_known_addr { host: "127.20.113.61" port: 41373 } attrs { promote: false } } peers { permanent_uuid: "e8ec8de97a86401e948b6ffb6a72e7d1" member_type: VOTER last_known_addr { host: "127.20.113.60" port: 42483 } attrs { promote: false } }
I20260806 09:29:16.100847 30827 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 377f07a0afaa4800aa98d87714775150 [term 1 FOLLOWER]: Refusing update from remote peer 0cbb5d09bd23462caa29c64dd33fdd25: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260806 09:29:16.100939 31107 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e8ec8de97a86401e948b6ffb6a72e7d1 [term 1 LEARNER]: Refusing update from remote peer 0cbb5d09bd23462caa29c64dd33fdd25: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260806 09:29:16.101511 31394 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0cbb5d09bd23462caa29c64dd33fdd25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "377f07a0afaa4800aa98d87714775150" member_type: VOTER last_known_addr { host: "127.20.113.61" port: 41373 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260806 09:29:16.101948 31393 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0cbb5d09bd23462caa29c64dd33fdd25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8ec8de97a86401e948b6ffb6a72e7d1" member_type: VOTER last_known_addr { host: "127.20.113.60" port: 42483 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260806 09:29:16.105335 31393 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0cbb5d09bd23462caa29c64dd33fdd25 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, e8ec8de97a86401e948b6ffb6a72e7d1 (127.20.113.60) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "0cbb5d09bd23462caa29c64dd33fdd25" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 45173 } } peers { permanent_uuid: "377f07a0afaa4800aa98d87714775150" member_type: VOTER last_known_addr { host: "127.20.113.61" port: 41373 } attrs { promote: false } } peers { permanent_uuid: "e8ec8de97a86401e948b6ffb6a72e7d1" member_type: VOTER last_known_addr { host: "127.20.113.60" port: 42483 } attrs { promote: false } } }
I20260806 09:29:16.106797 30827 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 377f07a0afaa4800aa98d87714775150 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, e8ec8de97a86401e948b6ffb6a72e7d1 (127.20.113.60) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "0cbb5d09bd23462caa29c64dd33fdd25" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 45173 } } peers { permanent_uuid: "377f07a0afaa4800aa98d87714775150" member_type: VOTER last_known_addr { host: "127.20.113.61" port: 41373 } attrs { promote: false } } peers { permanent_uuid: "e8ec8de97a86401e948b6ffb6a72e7d1" member_type: VOTER last_known_addr { host: "127.20.113.60" port: 42483 } attrs { promote: false } } }
I20260806 09:29:16.107076 31107 raft_consensus.cc:2932] T 00000000000000000000000000000000 P e8ec8de97a86401e948b6ffb6a72e7d1 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, e8ec8de97a86401e948b6ffb6a72e7d1 (127.20.113.60) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "0cbb5d09bd23462caa29c64dd33fdd25" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 45173 } } peers { permanent_uuid: "377f07a0afaa4800aa98d87714775150" member_type: VOTER last_known_addr { host: "127.20.113.61" port: 41373 } attrs { promote: false } } peers { permanent_uuid: "e8ec8de97a86401e948b6ffb6a72e7d1" member_type: VOTER last_known_addr { host: "127.20.113.60" port: 42483 } attrs { promote: false } } }
I20260806 09:29:16.109722 31393 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0cbb5d09bd23462caa29c64dd33fdd25 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "0cbb5d09bd23462caa29c64dd33fdd25" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "0cbb5d09bd23462caa29c64dd33fdd25" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 45173 } } peers { permanent_uuid: "377f07a0afaa4800aa98d87714775150" member_type: VOTER last_known_addr { host: "127.20.113.61" port: 41373 } attrs { promote: false } } peers { permanent_uuid: "e8ec8de97a86401e948b6ffb6a72e7d1" member_type: VOTER last_known_addr { host: "127.20.113.60" port: 42483 } attrs { promote: false } } }
I20260806 09:29:16.110008 31393 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0cbb5d09bd23462caa29c64dd33fdd25 [sys.catalog]: This master's current role is: LEADER
I20260806 09:29:16.110419 31170 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0cbb5d09bd23462caa29c64dd33fdd25 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0cbb5d09bd23462caa29c64dd33fdd25" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "0cbb5d09bd23462caa29c64dd33fdd25" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 45173 } } peers { permanent_uuid: "377f07a0afaa4800aa98d87714775150" member_type: VOTER last_known_addr { host: "127.20.113.61" port: 41373 } attrs { promote: false } } peers { permanent_uuid: "e8ec8de97a86401e948b6ffb6a72e7d1" member_type: VOTER last_known_addr { host: "127.20.113.60" port: 42483 } attrs { promote: false } } }
I20260806 09:29:16.110481 31393 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0cbb5d09bd23462caa29c64dd33fdd25 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0cbb5d09bd23462caa29c64dd33fdd25" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "0cbb5d09bd23462caa29c64dd33fdd25" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 45173 } } peers { permanent_uuid: "377f07a0afaa4800aa98d87714775150" member_type: VOTER last_known_addr { host: "127.20.113.61" port: 41373 } attrs { promote: false } } peers { permanent_uuid: "e8ec8de97a86401e948b6ffb6a72e7d1" member_type: VOTER last_known_addr { host: "127.20.113.60" port: 42483 } attrs { promote: false } } }
I20260806 09:29:16.110705 31170 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0cbb5d09bd23462caa29c64dd33fdd25 [sys.catalog]: This master's current role is: LEADER
I20260806 09:29:16.110743 31393 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0cbb5d09bd23462caa29c64dd33fdd25 [sys.catalog]: This master's current role is: LEADER
I20260806 09:29:16.116192 31401 sys_catalog.cc:455] T 00000000000000000000000000000000 P 377f07a0afaa4800aa98d87714775150 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0cbb5d09bd23462caa29c64dd33fdd25" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "0cbb5d09bd23462caa29c64dd33fdd25" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 45173 } } peers { permanent_uuid: "377f07a0afaa4800aa98d87714775150" member_type: VOTER last_known_addr { host: "127.20.113.61" port: 41373 } attrs { promote: false } } peers { permanent_uuid: "e8ec8de97a86401e948b6ffb6a72e7d1" member_type: VOTER last_known_addr { host: "127.20.113.60" port: 42483 } attrs { promote: false } } }
I20260806 09:29:16.116671 31401 sys_catalog.cc:458] T 00000000000000000000000000000000 P 377f07a0afaa4800aa98d87714775150 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 09:29:16.117261 31400 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8ec8de97a86401e948b6ffb6a72e7d1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0cbb5d09bd23462caa29c64dd33fdd25" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "0cbb5d09bd23462caa29c64dd33fdd25" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 45173 } } peers { permanent_uuid: "377f07a0afaa4800aa98d87714775150" member_type: VOTER last_known_addr { host: "127.20.113.61" port: 41373 } attrs { promote: false } } peers { permanent_uuid: "e8ec8de97a86401e948b6ffb6a72e7d1" member_type: VOTER last_known_addr { host: "127.20.113.60" port: 42483 } attrs { promote: false } } }
I20260806 09:29:16.117630 31400 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8ec8de97a86401e948b6ffb6a72e7d1 [sys.catalog]: This master's current role is: FOLLOWER
W20260806 09:29:16.175472 31391 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:29:16.175803 31391 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:29:16.175979 31391 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:29:16.183586 31391 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:29:16.183741 31391 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:29:16.183939 31391 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.113.2
I20260806 09:29:16.194363 31391 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:41701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/krb5kdc/oryx_127.20.113.2.keytab
--principal=oryx/127.20.113.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.113.2:38959
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.113.2
--webserver_port=46583
--tserver_master_addrs=127.20.113.62:45173,127.20.113.61:41373,127.20.113.60:42483
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.113.2
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:29:16.195802 31391 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:29:16.197353 31391 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:29:16.211390 31417 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:29:16.211525 31412 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:29:16.211426 31413 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:29:16.212600 31391 server_base.cc:1034] running on GCE node
I20260806 09:29:16.213249 31391 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:29:16.214346 31391 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:29:16.215590 31391 hybrid_clock.cc:648] HybridClock initialized: now 1786008556215519 us; error 58 us; skew 500 ppm
I20260806 09:29:16.215945 31391 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:29:16 dist-test-slave-45dx krb5kdc[29852](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008556, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.113.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:29:16.223690 31391 init.cc:377] Logged in from keytab as oryx/127.20.113.2@KRBTEST.COM (short username oryx)
I20260806 09:29:16.225286 31391 webserver.cc:460] Webserver started at http://127.20.113.2:46583/ using document root <none> and password file <none>
I20260806 09:29:16.226042 31391 fs_manager.cc:359] Metadata directory not provided
I20260806 09:29:16.226215 31391 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:29:16.232101 31391 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.000s
I20260806 09:29:16.235283 31425 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:29:16.236697 31391 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 09:29:16.236886 31391 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/ts-1/wal
uuid: "af2db7ab1bbc490f9bceae1317438337"
format_stamp: "Formatted at 2026-08-06 09:29:12 on dist-test-slave-45dx"
server_key: "7a04c34aded4b9ffb27c8afe515b7bc9"
server_key_iv: "fffd4978c7a08102e48407cf5479a707"
server_key_version: "encryptionkey@0"
I20260806 09:29:16.237920 31391 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:29:16.250645 31391 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:29:16.254755 31391 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:29:16.255075 31391 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:29:16.256099 31391 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:29:16.257558 31391 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:29:16.257678 31391 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:29:16.257812 31391 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:29:16.257915 31391 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:29:16.295634 31391 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.2:38959
I20260806 09:29:16.295720 31606 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.2:38959 every 8 connection(s)
I20260806 09:29:16.297196 31391 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/ts-1/data/info.pb
I20260806 09:29:16.299301 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 31391
I20260806 09:29:16.299640 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 30397
Aug 06 09:29:16 dist-test-slave-45dx krb5kdc[29852](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008556, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.113.2@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM
Aug 06 09:29:16 dist-test-slave-45dx krb5kdc[29852](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008556, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.113.2@KRBTEST.COM for oryx/127.20.113.60@KRBTEST.COM
Aug 06 09:29:16 dist-test-slave-45dx krb5kdc[29852](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008551, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.20.113.3@KRBTEST.COM
Aug 06 09:29:16 dist-test-slave-45dx krb5kdc[29852](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008556, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.113.2@KRBTEST.COM for oryx/127.20.113.61@KRBTEST.COM
I20260806 09:29:16.349867 31613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:16.300230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.2:35179 (local address 127.20.113.60:42483)
0806 09:29:16.325845 (+ 25615us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:16.325854 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:16.325877 (+ 23us) server_negotiation.cc:407] Connection header received
0806 09:29:16.325947 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:16.325954 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:16.326041 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:16.327481 (+ 1440us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:16.329148 (+ 1667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:16.330334 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:16.333309 (+ 2975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:16.334009 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:16.334137 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:16.341298 (+ 7161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:29:16.341305 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:29:16.341309 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:29:16.341377 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:29:16.345323 (+ 3946us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:16.346831 (+ 1508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:16.346839 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:16.346843 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:16.347011 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:16.347598 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:16.347603 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:16.347605 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:16.347967 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:29:16.348120 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:16.349164 (+ 1044us) server_negotiation.cc:299] Negotiation successful
0806 09:29:16.349446 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25478,"thread_start_us":25480,"threads_started":1}
I20260806 09:29:16.351542 31620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:16.300925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.60:42483 (local address 127.20.113.2:35179)
0806 09:29:16.310053 (+ 9128us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:16.310127 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:16.315006 (+ 4879us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:16.326527 (+ 11521us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:16.326542 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:16.327780 (+ 1238us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:29:16.328914 (+ 1134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:16.328945 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:16.330538 (+ 1593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:16.330544 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:16.333135 (+ 2591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:16.333150 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:16.335552 (+ 2402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:16.335564 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:16.335975 (+ 411us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:16.336970 (+ 995us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:29:16.337021 (+ 51us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:29:16.341065 (+ 4044us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:29:16.345513 (+ 4448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:29:16.345530 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:29:16.345549 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:29:16.346202 (+ 653us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:29:16.347185 (+ 983us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:29:16.347190 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:29:16.347194 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:29:16.347456 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:29:16.348216 (+ 760us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:29:16.348224 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:29:16.348712 (+ 488us) client_negotiation.cc:769] Sending connection context
0806 09:29:16.350097 (+ 1385us) client_negotiation.cc:240] Negotiation successful
0806 09:29:16.350443 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8718,"mutex_wait_us":381,"spinlock_wait_cycles":27776,"thread_start_us":23832,"threads_started":1}
I20260806 09:29:16.352245 31614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:16.300491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.2:47045 (local address 127.20.113.62:45173)
0806 09:29:16.324709 (+ 24218us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:16.324715 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:16.325336 (+ 621us) server_negotiation.cc:407] Connection header received
0806 09:29:16.326355 (+ 1019us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:16.326360 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:16.326449 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:16.326559 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:16.329143 (+ 2584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:16.330296 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:16.333305 (+ 3009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:16.334029 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:16.334126 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:16.341336 (+ 7210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:29:16.341343 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:29:16.341347 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:29:16.341400 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:29:16.345448 (+ 4048us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:16.347196 (+ 1748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:16.347202 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:16.347205 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:16.347409 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:16.347858 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:16.347867 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:16.347872 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:16.348264 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:29:16.348502 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:16.351607 (+ 3105us) server_negotiation.cc:299] Negotiation successful
0806 09:29:16.351844 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24071,"thread_start_us":24597,"threads_started":1}
I20260806 09:29:16.352751 31618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:16.300926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:45173 (local address 127.20.113.2:47045)
0806 09:29:16.325205 (+ 24279us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:16.325227 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:16.325369 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:16.326851 (+ 1482us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:16.326855 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:16.327594 (+ 739us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:29:16.328914 (+ 1320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:16.328945 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:16.330538 (+ 1593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:16.330544 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:16.333135 (+ 2591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:16.333150 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:16.335552 (+ 2402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:16.335564 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:16.335979 (+ 415us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:16.336990 (+ 1011us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:29:16.337019 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:29:16.340815 (+ 3796us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:29:16.346330 (+ 5515us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:29:16.346336 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:29:16.346340 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:29:16.347034 (+ 694us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:29:16.347549 (+ 515us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:29:16.347554 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:29:16.347558 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:29:16.347716 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:29:16.348553 (+ 837us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:29:16.348558 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:29:16.349127 (+ 569us) client_negotiation.cc:769] Sending connection context
0806 09:29:16.349332 (+ 205us) client_negotiation.cc:240] Negotiation successful
0806 09:29:16.349926 (+ 594us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24114,"mutex_wait_us":282,"spinlock_wait_cycles":28672,"thread_start_us":9286,"threads_started":1}
I20260806 09:29:16.352844 31616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:16.300826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37880 (local address 127.20.113.3:35129)
0806 09:29:16.311975 (+ 11149us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:16.311989 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:16.324743 (+ 12754us) server_negotiation.cc:407] Connection header received
0806 09:29:16.324955 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:16.324965 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:16.325275 (+ 310us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:16.325472 (+ 197us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:16.328099 (+ 2627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:16.330343 (+ 2244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:16.332949 (+ 2606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:16.334195 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:16.334327 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:16.344727 (+ 10400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:29:16.344747 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:29:16.344766 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:29:16.344814 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:29:16.348902 (+ 4088us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:16.349795 (+ 893us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:16.349804 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:16.349807 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:16.350097 (+ 290us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:16.350619 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:16.350625 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:16.350628 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:16.351170 (+ 542us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:29:16.351454 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:16.352101 (+ 647us) server_negotiation.cc:299] Negotiation successful
0806 09:29:16.352424 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10957,"thread_start_us":10797,"threads_started":1}
I20260806 09:29:16.354019 31608 heartbeater.cc:347] Connected to a master server at 127.20.113.62:45173
I20260806 09:29:16.354307 31608 heartbeater.cc:464] Registering TS with master...
I20260806 09:29:16.354421 30585 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:29:16.355232 31608 heartbeater.cc:511] Master 127.20.113.62:45173 requested a full tablet report, sending...
I20260806 09:29:16.356052 31607 heartbeater.cc:347] Connected to a master server at 127.20.113.60:42483
I20260806 09:29:16.356192 31607 heartbeater.cc:464] Registering TS with master...
I20260806 09:29:16.356673 31607 heartbeater.cc:511] Master 127.20.113.60:42483 requested a full tablet report, sending...
I20260806 09:29:16.356978 29915 ts_manager.cc:194] Re-registered known tserver with Master: af2db7ab1bbc490f9bceae1317438337 (127.20.113.2:38959)
I20260806 09:29:16.357342 31621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:16.325647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.2:42063 (local address 127.20.113.61:41373)
0806 09:29:16.326230 (+ 583us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:16.326236 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:16.335961 (+ 9725us) server_negotiation.cc:407] Connection header received
0806 09:29:16.336265 (+ 304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:16.336271 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:16.336393 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:16.336492 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:16.338481 (+ 1989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:16.339737 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:16.342102 (+ 2365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:16.342947 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:16.343101 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:16.347129 (+ 4028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:29:16.347137 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:29:16.347145 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:29:16.347192 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:29:16.351103 (+ 3911us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:16.353694 (+ 2591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:16.353699 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:16.353702 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:16.353872 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:16.354516 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:16.354524 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:16.354529 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:16.355015 (+ 486us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:29:16.355242 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:16.356105 (+ 863us) server_negotiation.cc:299] Negotiation successful
0806 09:29:16.356790 (+ 685us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":444,"thread_start_us":291,"threads_started":1}
I20260806 09:29:16.357867 31094 ts_manager.cc:194] Registered new tserver with Master: af2db7ab1bbc490f9bceae1317438337 (127.20.113.2:38959)
I20260806 09:29:16.358086 29915 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.20.113.2@KRBTEST.COM'} at 127.20.113.2:47045
I20260806 09:29:16.358691 31623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:16.326194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.61:41373 (local address 127.20.113.2:42063)
0806 09:29:16.335812 (+ 9618us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:16.335836 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:16.336096 (+ 260us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:16.336782 (+ 686us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:16.336789 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:16.337801 (+ 1012us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:29:16.338281 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:16.338297 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:16.339949 (+ 1652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:16.339958 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:16.341882 (+ 1924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:16.341911 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:16.343494 (+ 1583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:16.343502 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:16.343728 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:16.344361 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:29:16.344383 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:29:16.346900 (+ 2517us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:29:16.351270 (+ 4370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:29:16.351275 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:29:16.351278 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:29:16.353559 (+ 2281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:29:16.354033 (+ 474us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:29:16.354038 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:29:16.354041 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:29:16.354303 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:29:16.355380 (+ 1077us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:29:16.355386 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:29:16.355610 (+ 224us) client_negotiation.cc:769] Sending connection context
0806 09:29:16.357659 (+ 2049us) client_negotiation.cc:240] Negotiation successful
0806 09:29:16.358086 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9434,"thread_start_us":8195,"threads_started":1}
I20260806 09:29:16.359516 31609 heartbeater.cc:347] Connected to a master server at 127.20.113.61:41373
I20260806 09:29:16.359668 31609 heartbeater.cc:464] Registering TS with master...
I20260806 09:29:16.360467 31609 heartbeater.cc:511] Master 127.20.113.61:41373 requested a full tablet report, sending...
I20260806 09:29:16.361570 30815 ts_manager.cc:194] Registered new tserver with Master: af2db7ab1bbc490f9bceae1317438337 (127.20.113.2:38959)
I20260806 09:29:16.445972 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 30397
I20260806 09:29:16.470893 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.113.3:35129
--local_ip_for_outbound_sockets=127.20.113.3
--tserver_master_addrs=127.20.113.62:45173
--webserver_port=42243
--webserver_interface=127.20.113.3
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/krb5kdc/oryx_127.20.113.3.keytab
--principal=oryx/127.20.113.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:41701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.20.113.62:45173,127.20.113.61:41373
--tserver_master_addrs=127.20.113.62:45173,127.20.113.61:41373,127.20.113.60:42483 with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 09:29:16.720786 31372 heartbeater.cc:503] Master 127.20.113.62:45173 was elected leader, sending a full tablet report...
W20260806 09:29:16.825765 31627 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:29:16.826092 31627 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:29:16.826295 31627 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:29:16.833997 31627 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:29:16.834147 31627 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:29:16.834332 31627 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.113.3
I20260806 09:29:16.844944 31627 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:41701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/krb5kdc/oryx_127.20.113.3.keytab
--principal=oryx/127.20.113.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.113.3:35129
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.113.3
--webserver_port=42243
--tserver_master_addrs=127.20.113.62:45173,127.20.113.61:41373,127.20.113.60:42483
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.113.3
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:29:16.846352 31627 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:29:16.847700 31627 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:29:16.861375 31635 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:29:16.861328 31640 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:29:16.861301 31637 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:29:16.862560 31627 server_base.cc:1034] running on GCE node
I20260806 09:29:16.863173 31627 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:29:16.864135 31627 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:29:16.865365 31627 hybrid_clock.cc:648] HybridClock initialized: now 1786008556865317 us; error 58 us; skew 500 ppm
I20260806 09:29:16.865692 31627 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:29:16 dist-test-slave-45dx krb5kdc[29852](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008556, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.113.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:29:16.868950 31627 init.cc:377] Logged in from keytab as oryx/127.20.113.3@KRBTEST.COM (short username oryx)
I20260806 09:29:16.870548 31627 webserver.cc:460] Webserver started at http://127.20.113.3:42243/ using document root <none> and password file <none>
I20260806 09:29:16.871209 31627 fs_manager.cc:359] Metadata directory not provided
I20260806 09:29:16.871381 31627 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:29:16.876825 31627 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260806 09:29:16.879499 31653 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:29:16.880987 31627 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 09:29:16.881178 31627 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/ts-2/wal
uuid: "fb643e201d404a53a320740b57e36912"
format_stamp: "Formatted at 2026-08-06 09:29:13 on dist-test-slave-45dx"
server_key: "21113f96590945ac003474635bd5f5d1"
server_key_iv: "abbf4777e776df873a0427d3297b5519"
server_key_version: "encryptionkey@0"
I20260806 09:29:16.882233 31627 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:29:16.911230 31627 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:29:16.915475 31627 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:29:16.915798 31627 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:29:16.916857 31627 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:29:16.918241 31627 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:29:16.918380 31627 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:29:16.918512 31627 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:29:16.918606 31627 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:29:16.953063 31627 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.3:35129
I20260806 09:29:16.953153 31818 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.3:35129 every 8 connection(s)
I20260806 09:29:16.954545 31627 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/ts-2/data/info.pb
I20260806 09:29:16.959915 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 31627
Aug 06 09:29:16 dist-test-slave-45dx krb5kdc[29852](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008556, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.113.3@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM
Aug 06 09:29:16 dist-test-slave-45dx krb5kdc[29852](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008556, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.113.3@KRBTEST.COM for oryx/127.20.113.60@KRBTEST.COM
I20260806 09:29:16.986174 31829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:16.958039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.3:45497 (local address 127.20.113.62:45173)
0806 09:29:16.958547 (+ 508us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:16.958552 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:16.959946 (+ 1394us) server_negotiation.cc:407] Connection header received
0806 09:29:16.963971 (+ 4025us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:16.963982 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:16.964115 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:16.964218 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:16.967665 (+ 3447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:16.969164 (+ 1499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:16.972200 (+ 3036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:16.972863 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:16.972989 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:16.978481 (+ 5492us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:29:16.978486 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:29:16.978489 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:29:16.978525 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:29:16.981877 (+ 3352us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:16.982953 (+ 1076us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:16.982961 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:16.982968 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:16.983141 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:16.983758 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:16.983764 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:16.983767 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:16.984210 (+ 443us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:29:16.984386 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:16.985503 (+ 1117us) server_negotiation.cc:299] Negotiation successful
0806 09:29:16.985737 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":384,"thread_start_us":263,"threads_started":1}
I20260806 09:29:16.988832 31831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:16.959016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:45173 (local address 127.20.113.3:45497)
0806 09:29:16.959844 (+ 828us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:16.959860 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:16.963738 (+ 3878us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:16.964896 (+ 1158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:16.964914 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:16.966247 (+ 1333us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:29:16.967379 (+ 1132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:16.967413 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:16.969455 (+ 2042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:16.969463 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:16.972008 (+ 2545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:16.972023 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:16.973368 (+ 1345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:16.973377 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:16.973669 (+ 292us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:16.974639 (+ 970us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:29:16.974685 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:29:16.978167 (+ 3482us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:29:16.982058 (+ 3891us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:29:16.982070 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:29:16.982089 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:29:16.982769 (+ 680us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:29:16.983309 (+ 540us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:29:16.983313 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:29:16.983317 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:29:16.983609 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:29:16.984513 (+ 904us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:29:16.984523 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:29:16.985027 (+ 504us) client_negotiation.cc:769] Sending connection context
0806 09:29:16.986723 (+ 1696us) client_negotiation.cc:240] Negotiation successful
0806 09:29:16.987285 (+ 562us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":695,"thread_start_us":1815,"threads_started":1}
I20260806 09:29:16.989315 31830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:16.958377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.60:42483 (local address 127.20.113.3:46615)
0806 09:29:16.959590 (+ 1213us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:16.959689 (+ 99us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:16.963228 (+ 3539us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:16.971952 (+ 8724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:16.971957 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:16.972542 (+ 585us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:29:16.972827 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:16.972839 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:16.974555 (+ 1716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:16.974560 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:16.976259 (+ 1699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:16.976277 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:16.977127 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:16.977135 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:16.977786 (+ 651us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:16.979097 (+ 1311us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:29:16.979113 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:29:16.981954 (+ 2841us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:29:16.986387 (+ 4433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:29:16.986392 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:29:16.986394 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:29:16.986850 (+ 456us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:29:16.987310 (+ 460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:29:16.987314 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:29:16.987317 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:29:16.987521 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:29:16.988196 (+ 675us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:29:16.988201 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:29:16.988380 (+ 179us) client_negotiation.cc:769] Sending connection context
0806 09:29:16.988516 (+ 136us) client_negotiation.cc:240] Negotiation successful
0806 09:29:16.988801 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":763,"spinlock_wait_cycles":51968,"thread_start_us":744,"threads_started":1}
I20260806 09:29:16.990505 31825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:16.957672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.3:46615 (local address 127.20.113.60:42483)
0806 09:29:16.971452 (+ 13780us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:16.971459 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:16.971480 (+ 21us) server_negotiation.cc:407] Connection header received
0806 09:29:16.971543 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:16.971548 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:16.971653 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:16.973001 (+ 1348us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:16.973190 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:16.974272 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:16.976469 (+ 2197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:16.976907 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:16.977065 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:16.982179 (+ 5114us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:29:16.982185 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:29:16.982191 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:29:16.982233 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:29:16.986229 (+ 3996us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:16.987010 (+ 781us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:16.987015 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:16.987019 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:16.987175 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:16.987680 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:16.987685 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:16.987687 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:16.988052 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:29:16.989756 (+ 1704us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:16.989883 (+ 127us) server_negotiation.cc:299] Negotiation successful
0806 09:29:16.990121 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13608,"thread_start_us":14075,"threads_started":1}
I20260806 09:29:16.990993 31821 heartbeater.cc:347] Connected to a master server at 127.20.113.62:45173
I20260806 09:29:16.991351 31821 heartbeater.cc:464] Registering TS with master...
I20260806 09:29:16.991587 31819 heartbeater.cc:347] Connected to a master server at 127.20.113.60:42483
I20260806 09:29:16.991757 31819 heartbeater.cc:464] Registering TS with master...
I20260806 09:29:16.992203 31821 heartbeater.cc:511] Master 127.20.113.62:45173 requested a full tablet report, sending...
I20260806 09:29:16.992369 31819 heartbeater.cc:511] Master 127.20.113.60:42483 requested a full tablet report, sending...
I20260806 09:29:16.993968 29915 ts_manager.cc:194] Re-registered known tserver with Master: fb643e201d404a53a320740b57e36912 (127.20.113.3:35129)
I20260806 09:29:16.994177 31094 ts_manager.cc:194] Registered new tserver with Master: fb643e201d404a53a320740b57e36912 (127.20.113.3:35129)
I20260806 09:29:16.995275 29915 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.20.113.3@KRBTEST.COM'} at 127.20.113.3:45497
Aug 06 09:29:16 dist-test-slave-45dx krb5kdc[29852](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008556, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.113.3@KRBTEST.COM for oryx/127.20.113.61@KRBTEST.COM
I20260806 09:29:17.009359 31833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:16.961228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.61:41373 (local address 127.20.113.3:55473)
0806 09:29:16.982181 (+ 20953us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:16.982207 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:16.982386 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:16.983434 (+ 1048us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:16.983442 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:16.984212 (+ 770us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:29:16.984726 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:16.984742 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:16.987386 (+ 2644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:16.987394 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:16.989828 (+ 2434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:16.989847 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:16.992976 (+ 3129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:16.992986 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:16.993259 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:16.995013 (+ 1754us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:29:16.995044 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:29:16.997874 (+ 2830us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:29:17.006299 (+ 8425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:29:17.006305 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:29:17.006308 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:29:17.006826 (+ 518us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:29:17.007295 (+ 469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:29:17.007302 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:29:17.007307 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:29:17.007497 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:29:17.008260 (+ 763us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:29:17.008265 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:29:17.008457 (+ 192us) client_negotiation.cc:769] Sending connection context
0806 09:29:17.008586 (+ 129us) client_negotiation.cc:240] Negotiation successful
0806 09:29:17.008924 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20767,"thread_start_us":21267,"threads_started":1}
I20260806 09:29:17.010814 31832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:16.960680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.3:55473 (local address 127.20.113.61:41373)
0806 09:29:16.981529 (+ 20849us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:16.981538 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:16.982845 (+ 1307us) server_negotiation.cc:407] Connection header received
0806 09:29:16.982944 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:16.982952 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:16.983074 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:16.983198 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:16.984946 (+ 1748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:16.986774 (+ 1828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:16.990035 (+ 3261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:16.990907 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:16.991358 (+ 451us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:16.998089 (+ 6731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:29:16.998096 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:29:16.998101 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:29:16.998152 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:29:17.006061 (+ 7909us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:17.006991 (+ 930us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:17.006995 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:17.007000 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:17.007158 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:17.007643 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:17.007648 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:17.007651 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:17.008118 (+ 467us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:29:17.010040 (+ 1922us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:17.010179 (+ 139us) server_negotiation.cc:299] Negotiation successful
0806 09:29:17.010422 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20666,"thread_start_us":22029,"threads_started":1}
I20260806 09:29:17.012022 31822 heartbeater.cc:347] Connected to a master server at 127.20.113.61:41373
I20260806 09:29:17.012184 31822 heartbeater.cc:464] Registering TS with master...
I20260806 09:29:17.012704 31822 heartbeater.cc:511] Master 127.20.113.61:41373 requested a full tablet report, sending...
I20260806 09:29:17.014008 30815 ts_manager.cc:194] Registered new tserver with Master: fb643e201d404a53a320740b57e36912 (127.20.113.3:35129)
I20260806 09:29:17.020186 31825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:17.001227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60292 (local address 127.20.113.60:42483)
0806 09:29:17.001380 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:17.001385 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:17.002259 (+ 874us) server_negotiation.cc:407] Connection header received
0806 09:29:17.002476 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:17.002481 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:17.002554 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:17.002657 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:17.004712 (+ 2055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:17.005829 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:17.008337 (+ 2508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:17.008973 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:17.009082 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:17.013176 (+ 4094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:29:17.013186 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:29:17.013192 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:29:17.013230 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:29:17.017146 (+ 3916us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:17.017960 (+ 814us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:17.017967 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:17.017973 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:17.018138 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:17.018581 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:17.018586 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:17.018590 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:17.018915 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:29:17.019092 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:17.019673 (+ 581us) server_negotiation.cc:299] Negotiation successful
0806 09:29:17.019907 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 09:29:17.020553 31829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:16.999770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59218 (local address 127.20.113.62:45173)
0806 09:29:16.999941 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:16.999949 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:17.000852 (+ 903us) server_negotiation.cc:407] Connection header received
0806 09:29:17.001173 (+ 321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:17.001182 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:17.001293 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:17.001373 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:17.004877 (+ 3504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:17.005966 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:17.008102 (+ 2136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:17.008779 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:17.008909 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:17.013303 (+ 4394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:29:17.013309 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:29:17.013313 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:29:17.013352 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:29:17.017089 (+ 3737us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:17.018166 (+ 1077us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:17.018172 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:17.018175 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:17.018331 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:17.018786 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:17.018791 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:17.018794 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:17.019161 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:29:17.019401 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:17.019744 (+ 343us) server_negotiation.cc:299] Negotiation successful
0806 09:29:17.020160 (+ 416us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 09:29:17.023108 31841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:16.999802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43078 (local address 127.20.113.61:41373)
0806 09:29:17.004824 (+ 5022us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:17.004829 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:17.004845 (+ 16us) server_negotiation.cc:407] Connection header received
0806 09:29:17.004892 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:17.004896 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:17.004977 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:17.007470 (+ 2493us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:17.007712 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:17.009013 (+ 1301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:17.011457 (+ 2444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:17.012222 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:17.012438 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:17.016167 (+ 3729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:29:17.016176 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:29:17.016179 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:29:17.016207 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:29:17.019514 (+ 3307us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:17.020484 (+ 970us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:17.020488 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:17.020492 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:17.020643 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:17.021328 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:17.021333 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:17.021336 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:17.021674 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:29:17.021960 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:17.022403 (+ 443us) server_negotiation.cc:299] Negotiation successful
0806 09:29:17.022621 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4880,"thread_start_us":6341,"threads_started":1}
I20260806 09:29:17.025233 29915 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59218:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 09:29:17.032029 29915 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 09:29:17.073495 31860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:17.054536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37894 (local address 127.20.113.3:35129)
0806 09:29:17.066101 (+ 11565us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:17.066110 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:17.066142 (+ 32us) server_negotiation.cc:407] Connection header received
0806 09:29:17.066241 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:17.066246 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:17.066460 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:17.066680 (+ 220us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:29:17.067450 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:17.068761 (+ 1311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:17.070992 (+ 2231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:17.072528 (+ 1536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:17.072675 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:17.072895 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:17.073071 (+ 176us) server_negotiation.cc:299] Negotiation successful
0806 09:29:17.073224 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11395,"thread_start_us":11203,"threads_started":1}
I20260806 09:29:17.073532 31858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:17.054256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.3:35129 (local address 127.0.0.1:37894)
0806 09:29:17.054907 (+ 651us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:17.054932 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:17.056673 (+ 1741us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:17.066670 (+ 9997us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:17.066674 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:17.066697 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:29:17.067133 (+ 436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:17.067148 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:17.068916 (+ 1768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:17.068925 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:17.070818 (+ 1893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:17.070841 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:17.072694 (+ 1853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:17.072701 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:17.072873 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:17.072907 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 09:29:17.072992 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 09:29:17.073123 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"mutex_wait_us":1,"thread_start_us":1828,"threads_started":1}
I20260806 09:29:17.074293 31867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:17.054756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.2:38959 (local address 127.0.0.1:44834)
0806 09:29:17.064863 (+ 10107us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:17.064887 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:17.065167 (+ 280us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:17.065984 (+ 817us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:17.065992 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:17.066019 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:29:17.066268 (+ 249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:17.066276 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:17.068096 (+ 1820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:17.068101 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:17.070347 (+ 2246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:17.070362 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:17.071774 (+ 1412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:17.071779 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:17.072181 (+ 402us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:17.072215 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 09:29:17.072331 (+ 116us) client_negotiation.cc:240] Negotiation successful
0806 09:29:17.072463 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9941,"mutex_wait_us":220,"thread_start_us":10684,"threads_started":1}
I20260806 09:29:17.075117 31864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:17.055915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44834 (local address 127.20.113.2:38959)
0806 09:29:17.060938 (+ 5023us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:17.060951 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:17.065029 (+ 4078us) server_negotiation.cc:407] Connection header received
0806 09:29:17.065414 (+ 385us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:17.065427 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:17.065736 (+ 309us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:17.065981 (+ 245us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:29:17.066416 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:17.067939 (+ 1523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:17.070551 (+ 2612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:17.071613 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:17.071754 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:17.071992 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:17.074610 (+ 2618us) server_negotiation.cc:299] Negotiation successful
0806 09:29:17.074818 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4729,"thread_start_us":4584,"threads_started":1}
I20260806 09:29:17.075196 31868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:17.056710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36242 (local address 127.20.113.1:37689)
0806 09:29:17.057429 (+ 719us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:17.057449 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:17.065881 (+ 8432us) server_negotiation.cc:407] Connection header received
0806 09:29:17.066615 (+ 734us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:17.066624 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:17.066856 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:17.067095 (+ 239us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:29:17.068069 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:17.069685 (+ 1616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:17.072118 (+ 2433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:17.073914 (+ 1796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:17.074060 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:17.074287 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:17.074776 (+ 489us) server_negotiation.cc:299] Negotiation successful
0806 09:29:17.074918 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":498,"thread_start_us":336,"threads_started":1}
I20260806 09:29:17.075840 31862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:17.056156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.1:37689 (local address 127.0.0.1:36242)
0806 09:29:17.065771 (+ 9615us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:17.065789 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:17.066413 (+ 624us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:17.067337 (+ 924us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:17.067340 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:17.067384 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:29:17.067805 (+ 421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:17.067818 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:17.069855 (+ 2037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:17.069860 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:17.071957 (+ 2097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:17.071975 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:17.074184 (+ 2209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:17.074192 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:17.074534 (+ 342us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:17.074570 (+ 36us) client_negotiation.cc:769] Sending connection context
0806 09:29:17.075457 (+ 887us) client_negotiation.cc:240] Negotiation successful
0806 09:29:17.075573 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9461,"mutex_wait_us":1,"thread_start_us":8354,"threads_started":1}
I20260806 09:29:17.077715 31714 tablet_service.cc:1467] Processing CreateTablet for tablet e6bcc8de9cdc453d8e0f2414e7e20d07 (DEFAULT_TABLE table=test-table [id=f37e78b276324b6e9a42238136572c64]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:29:17.079311 31714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6bcc8de9cdc453d8e0f2414e7e20d07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:29:17.079675 31502 tablet_service.cc:1467] Processing CreateTablet for tablet e6bcc8de9cdc453d8e0f2414e7e20d07 (DEFAULT_TABLE table=test-table [id=f37e78b276324b6e9a42238136572c64]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:29:17.079675 31261 tablet_service.cc:1467] Processing CreateTablet for tablet e6bcc8de9cdc453d8e0f2414e7e20d07 (DEFAULT_TABLE table=test-table [id=f37e78b276324b6e9a42238136572c64]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:29:17.081187 31502 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6bcc8de9cdc453d8e0f2414e7e20d07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:29:17.081215 31261 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6bcc8de9cdc453d8e0f2414e7e20d07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:29:17.092636 31869 tablet_bootstrap.cc:492] T e6bcc8de9cdc453d8e0f2414e7e20d07 P fb643e201d404a53a320740b57e36912: Bootstrap starting.
I20260806 09:29:17.097152 31869 tablet_bootstrap.cc:654] T e6bcc8de9cdc453d8e0f2414e7e20d07 P fb643e201d404a53a320740b57e36912: Neither blocks nor log segments found. Creating new log.
I20260806 09:29:17.097517 31873 tablet_bootstrap.cc:492] T e6bcc8de9cdc453d8e0f2414e7e20d07 P f151cc51113947f6a0ea2f463ed20681: Bootstrap starting.
I20260806 09:29:17.099131 31869 log.cc:824] T e6bcc8de9cdc453d8e0f2414e7e20d07 P fb643e201d404a53a320740b57e36912: Log is configured to *not* fsync() on all Append() calls
I20260806 09:29:17.100911 31870 tablet_bootstrap.cc:492] T e6bcc8de9cdc453d8e0f2414e7e20d07 P af2db7ab1bbc490f9bceae1317438337: Bootstrap starting.
I20260806 09:29:17.101641 31873 tablet_bootstrap.cc:654] T e6bcc8de9cdc453d8e0f2414e7e20d07 P f151cc51113947f6a0ea2f463ed20681: Neither blocks nor log segments found. Creating new log.
I20260806 09:29:17.103005 31869 tablet_bootstrap.cc:492] T e6bcc8de9cdc453d8e0f2414e7e20d07 P fb643e201d404a53a320740b57e36912: No bootstrap required, opened a new log
I20260806 09:29:17.103088 31873 log.cc:824] T e6bcc8de9cdc453d8e0f2414e7e20d07 P f151cc51113947f6a0ea2f463ed20681: Log is configured to *not* fsync() on all Append() calls
I20260806 09:29:17.103348 31869 ts_tablet_manager.cc:1397] T e6bcc8de9cdc453d8e0f2414e7e20d07 P fb643e201d404a53a320740b57e36912: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.000s
I20260806 09:29:17.104883 31870 tablet_bootstrap.cc:654] T e6bcc8de9cdc453d8e0f2414e7e20d07 P af2db7ab1bbc490f9bceae1317438337: Neither blocks nor log segments found. Creating new log.
I20260806 09:29:17.106201 31870 log.cc:824] T e6bcc8de9cdc453d8e0f2414e7e20d07 P af2db7ab1bbc490f9bceae1317438337: Log is configured to *not* fsync() on all Append() calls
I20260806 09:29:17.106765 31873 tablet_bootstrap.cc:492] T e6bcc8de9cdc453d8e0f2414e7e20d07 P f151cc51113947f6a0ea2f463ed20681: No bootstrap required, opened a new log
I20260806 09:29:17.107075 31873 ts_tablet_manager.cc:1397] T e6bcc8de9cdc453d8e0f2414e7e20d07 P f151cc51113947f6a0ea2f463ed20681: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260806 09:29:17.111289 31870 tablet_bootstrap.cc:492] T e6bcc8de9cdc453d8e0f2414e7e20d07 P af2db7ab1bbc490f9bceae1317438337: No bootstrap required, opened a new log
I20260806 09:29:17.111586 31870 ts_tablet_manager.cc:1397] T e6bcc8de9cdc453d8e0f2414e7e20d07 P af2db7ab1bbc490f9bceae1317438337: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.005s
I20260806 09:29:17.112870 31869 raft_consensus.cc:348] T e6bcc8de9cdc453d8e0f2414e7e20d07 P fb643e201d404a53a320740b57e36912 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb643e201d404a53a320740b57e36912" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 35129 } } peers { permanent_uuid: "af2db7ab1bbc490f9bceae1317438337" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 38959 } } peers { permanent_uuid: "f151cc51113947f6a0ea2f463ed20681" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 37689 } }
I20260806 09:29:17.113261 31869 raft_consensus.cc:374] T e6bcc8de9cdc453d8e0f2414e7e20d07 P fb643e201d404a53a320740b57e36912 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:29:17.113391 31869 raft_consensus.cc:729] T e6bcc8de9cdc453d8e0f2414e7e20d07 P fb643e201d404a53a320740b57e36912 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb643e201d404a53a320740b57e36912, State: Initialized, Role: FOLLOWER
I20260806 09:29:17.113979 31869 consensus_queue.cc:260] T e6bcc8de9cdc453d8e0f2414e7e20d07 P fb643e201d404a53a320740b57e36912 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb643e201d404a53a320740b57e36912" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 35129 } } peers { permanent_uuid: "af2db7ab1bbc490f9bceae1317438337" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 38959 } } peers { permanent_uuid: "f151cc51113947f6a0ea2f463ed20681" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 37689 } }
I20260806 09:29:17.116151 31821 heartbeater.cc:503] Master 127.20.113.62:45173 was elected leader, sending a full tablet report...
I20260806 09:29:17.117708 31873 raft_consensus.cc:348] T e6bcc8de9cdc453d8e0f2414e7e20d07 P f151cc51113947f6a0ea2f463ed20681 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb643e201d404a53a320740b57e36912" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 35129 } } peers { permanent_uuid: "af2db7ab1bbc490f9bceae1317438337" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 38959 } } peers { permanent_uuid: "f151cc51113947f6a0ea2f463ed20681" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 37689 } }
I20260806 09:29:17.118160 31873 raft_consensus.cc:374] T e6bcc8de9cdc453d8e0f2414e7e20d07 P f151cc51113947f6a0ea2f463ed20681 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:29:17.118283 31873 raft_consensus.cc:729] T e6bcc8de9cdc453d8e0f2414e7e20d07 P f151cc51113947f6a0ea2f463ed20681 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f151cc51113947f6a0ea2f463ed20681, State: Initialized, Role: FOLLOWER
I20260806 09:29:17.119061 31869 ts_tablet_manager.cc:1428] T e6bcc8de9cdc453d8e0f2414e7e20d07 P fb643e201d404a53a320740b57e36912: Time spent starting tablet: real 0.016s user 0.013s sys 0.000s
I20260806 09:29:17.118914 31873 consensus_queue.cc:260] T e6bcc8de9cdc453d8e0f2414e7e20d07 P f151cc51113947f6a0ea2f463ed20681 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb643e201d404a53a320740b57e36912" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 35129 } } peers { permanent_uuid: "af2db7ab1bbc490f9bceae1317438337" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 38959 } } peers { permanent_uuid: "f151cc51113947f6a0ea2f463ed20681" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 37689 } }
I20260806 09:29:17.131395 31870 raft_consensus.cc:348] T e6bcc8de9cdc453d8e0f2414e7e20d07 P af2db7ab1bbc490f9bceae1317438337 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb643e201d404a53a320740b57e36912" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 35129 } } peers { permanent_uuid: "af2db7ab1bbc490f9bceae1317438337" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 38959 } } peers { permanent_uuid: "f151cc51113947f6a0ea2f463ed20681" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 37689 } }
I20260806 09:29:17.131892 31870 raft_consensus.cc:374] T e6bcc8de9cdc453d8e0f2414e7e20d07 P af2db7ab1bbc490f9bceae1317438337 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:29:17.132016 31870 raft_consensus.cc:729] T e6bcc8de9cdc453d8e0f2414e7e20d07 P af2db7ab1bbc490f9bceae1317438337 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af2db7ab1bbc490f9bceae1317438337, State: Initialized, Role: FOLLOWER
I20260806 09:29:17.132287 31873 ts_tablet_manager.cc:1428] T e6bcc8de9cdc453d8e0f2414e7e20d07 P f151cc51113947f6a0ea2f463ed20681: Time spent starting tablet: real 0.025s user 0.018s sys 0.005s
I20260806 09:29:17.132767 31870 consensus_queue.cc:260] T e6bcc8de9cdc453d8e0f2414e7e20d07 P af2db7ab1bbc490f9bceae1317438337 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb643e201d404a53a320740b57e36912" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 35129 } } peers { permanent_uuid: "af2db7ab1bbc490f9bceae1317438337" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 38959 } } peers { permanent_uuid: "f151cc51113947f6a0ea2f463ed20681" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 37689 } }
I20260806 09:29:17.135020 31608 heartbeater.cc:503] Master 127.20.113.62:45173 was elected leader, sending a full tablet report...
I20260806 09:29:17.135851 31870 ts_tablet_manager.cc:1428] T e6bcc8de9cdc453d8e0f2414e7e20d07 P af2db7ab1bbc490f9bceae1317438337: Time spent starting tablet: real 0.024s user 0.013s sys 0.005s
W20260806 09:29:17.182972 31374 tablet.cc:2369] T e6bcc8de9cdc453d8e0f2414e7e20d07 P f151cc51113947f6a0ea2f463ed20681: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 09:29:17.208503 31824 tablet.cc:2369] T e6bcc8de9cdc453d8e0f2414e7e20d07 P fb643e201d404a53a320740b57e36912: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 09:29:17.315325 31612 tablet.cc:2369] T e6bcc8de9cdc453d8e0f2414e7e20d07 P af2db7ab1bbc490f9bceae1317438337: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 09:29:17.550848 31882 raft_consensus.cc:482] T e6bcc8de9cdc453d8e0f2414e7e20d07 P af2db7ab1bbc490f9bceae1317438337 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 09:29:17.551183 31882 raft_consensus.cc:504] T e6bcc8de9cdc453d8e0f2414e7e20d07 P af2db7ab1bbc490f9bceae1317438337 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb643e201d404a53a320740b57e36912" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 35129 } } peers { permanent_uuid: "af2db7ab1bbc490f9bceae1317438337" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 38959 } } peers { permanent_uuid: "f151cc51113947f6a0ea2f463ed20681" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 37689 } }
I20260806 09:29:17.553056 31882 leader_election.cc:288] T e6bcc8de9cdc453d8e0f2414e7e20d07 P af2db7ab1bbc490f9bceae1317438337 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fb643e201d404a53a320740b57e36912 (127.20.113.3:35129), f151cc51113947f6a0ea2f463ed20681 (127.20.113.1:37689)
I20260806 09:29:17.559767 31860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:17.553376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.2:49877 (local address 127.20.113.3:35129)
0806 09:29:17.553628 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:17.553636 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:17.554434 (+ 798us) server_negotiation.cc:407] Connection header received
0806 09:29:17.554694 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:17.554697 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:17.554771 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:17.554881 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:29:17.555471 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:17.556156 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:17.558023 (+ 1867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:17.558753 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:17.558936 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:17.559012 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:17.559365 (+ 353us) server_negotiation.cc:299] Negotiation successful
0806 09:29:17.559497 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260806 09:29:17.559839 31868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:17.553696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.2:51951 (local address 127.20.113.1:37689)
0806 09:29:17.553888 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:17.553895 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:17.554569 (+ 674us) server_negotiation.cc:407] Connection header received
0806 09:29:17.554801 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:17.554805 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:17.554882 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:17.554986 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:29:17.555588 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:17.556271 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:17.558027 (+ 1756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:17.558786 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:17.558975 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:17.559050 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:17.559405 (+ 355us) server_negotiation.cc:299] Negotiation successful
0806 09:29:17.559542 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 09:29:17.559922 31884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:17.553295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.3:35129 (local address 127.20.113.2:49877)
0806 09:29:17.554295 (+ 1000us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:17.554318 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:17.554513 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:17.554899 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:17.554903 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:17.554928 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:29:17.555312 (+ 384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:17.555322 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:17.556319 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:17.556324 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:17.557861 (+ 1537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:17.557873 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:17.558967 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:17.558976 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:17.559185 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:17.559222 (+ 37us) client_negotiation.cc:769] Sending connection context
0806 09:29:17.559337 (+ 115us) client_negotiation.cc:240] Negotiation successful
0806 09:29:17.559497 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":833,"mutex_wait_us":87,"thread_start_us":852,"threads_started":1}
I20260806 09:29:17.560408 31883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:17.553366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.1:37689 (local address 127.20.113.2:51951)
0806 09:29:17.554450 (+ 1084us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:17.554465 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:17.554622 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:17.555006 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:17.555010 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:17.555039 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:29:17.555417 (+ 378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:17.555427 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:17.556460 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:17.556466 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:17.557862 (+ 1396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:17.557874 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:17.559016 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:17.559023 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:17.559234 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:17.559256 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 09:29:17.559378 (+ 122us) client_negotiation.cc:240] Negotiation successful
0806 09:29:17.559541 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":952,"mutex_wait_us":77,"thread_start_us":556,"threads_started":1}
I20260806 09:29:17.560998 31755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6bcc8de9cdc453d8e0f2414e7e20d07" candidate_uuid: "af2db7ab1bbc490f9bceae1317438337" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb643e201d404a53a320740b57e36912" is_pre_election: true
I20260806 09:29:17.561452 31755 raft_consensus.cc:2454] T e6bcc8de9cdc453d8e0f2414e7e20d07 P fb643e201d404a53a320740b57e36912 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate af2db7ab1bbc490f9bceae1317438337 in term 0.
I20260806 09:29:17.561378 31288 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6bcc8de9cdc453d8e0f2414e7e20d07" candidate_uuid: "af2db7ab1bbc490f9bceae1317438337" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f151cc51113947f6a0ea2f463ed20681" is_pre_election: true
I20260806 09:29:17.561887 31288 raft_consensus.cc:2454] T e6bcc8de9cdc453d8e0f2414e7e20d07 P f151cc51113947f6a0ea2f463ed20681 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate af2db7ab1bbc490f9bceae1317438337 in term 0.
I20260806 09:29:17.562103 31429 leader_election.cc:302] T e6bcc8de9cdc453d8e0f2414e7e20d07 P af2db7ab1bbc490f9bceae1317438337 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: af2db7ab1bbc490f9bceae1317438337, fb643e201d404a53a320740b57e36912; no voters:
I20260806 09:29:17.562516 31882 raft_consensus.cc:2781] T e6bcc8de9cdc453d8e0f2414e7e20d07 P af2db7ab1bbc490f9bceae1317438337 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 09:29:17.562683 31882 raft_consensus.cc:482] T e6bcc8de9cdc453d8e0f2414e7e20d07 P af2db7ab1bbc490f9bceae1317438337 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 09:29:17.562804 31882 raft_consensus.cc:3037] T e6bcc8de9cdc453d8e0f2414e7e20d07 P af2db7ab1bbc490f9bceae1317438337 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:29:17.565099 31882 raft_consensus.cc:504] T e6bcc8de9cdc453d8e0f2414e7e20d07 P af2db7ab1bbc490f9bceae1317438337 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb643e201d404a53a320740b57e36912" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 35129 } } peers { permanent_uuid: "af2db7ab1bbc490f9bceae1317438337" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 38959 } } peers { permanent_uuid: "f151cc51113947f6a0ea2f463ed20681" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 37689 } }
I20260806 09:29:17.565666 31882 leader_election.cc:288] T e6bcc8de9cdc453d8e0f2414e7e20d07 P af2db7ab1bbc490f9bceae1317438337 [CANDIDATE]: Term 1 election: Requested vote from peers fb643e201d404a53a320740b57e36912 (127.20.113.3:35129), f151cc51113947f6a0ea2f463ed20681 (127.20.113.1:37689)
I20260806 09:29:17.565999 31755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6bcc8de9cdc453d8e0f2414e7e20d07" candidate_uuid: "af2db7ab1bbc490f9bceae1317438337" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb643e201d404a53a320740b57e36912"
I20260806 09:29:17.566147 31288 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6bcc8de9cdc453d8e0f2414e7e20d07" candidate_uuid: "af2db7ab1bbc490f9bceae1317438337" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f151cc51113947f6a0ea2f463ed20681"
I20260806 09:29:17.566237 31755 raft_consensus.cc:3037] T e6bcc8de9cdc453d8e0f2414e7e20d07 P fb643e201d404a53a320740b57e36912 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:29:17.566380 31288 raft_consensus.cc:3037] T e6bcc8de9cdc453d8e0f2414e7e20d07 P f151cc51113947f6a0ea2f463ed20681 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:29:17.568465 31755 raft_consensus.cc:2454] T e6bcc8de9cdc453d8e0f2414e7e20d07 P fb643e201d404a53a320740b57e36912 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af2db7ab1bbc490f9bceae1317438337 in term 1.
I20260806 09:29:17.568707 31288 raft_consensus.cc:2454] T e6bcc8de9cdc453d8e0f2414e7e20d07 P f151cc51113947f6a0ea2f463ed20681 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af2db7ab1bbc490f9bceae1317438337 in term 1.
I20260806 09:29:17.568876 31429 leader_election.cc:302] T e6bcc8de9cdc453d8e0f2414e7e20d07 P af2db7ab1bbc490f9bceae1317438337 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: af2db7ab1bbc490f9bceae1317438337, fb643e201d404a53a320740b57e36912; no voters:
I20260806 09:29:17.569178 31882 raft_consensus.cc:2781] T e6bcc8de9cdc453d8e0f2414e7e20d07 P af2db7ab1bbc490f9bceae1317438337 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:29:17.569590 31882 raft_consensus.cc:686] T e6bcc8de9cdc453d8e0f2414e7e20d07 P af2db7ab1bbc490f9bceae1317438337 [term 1 LEADER]: Becoming Leader. State: Replica: af2db7ab1bbc490f9bceae1317438337, State: Running, Role: LEADER
I20260806 09:29:17.570158 31882 consensus_queue.cc:237] T e6bcc8de9cdc453d8e0f2414e7e20d07 P af2db7ab1bbc490f9bceae1317438337 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb643e201d404a53a320740b57e36912" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 35129 } } peers { permanent_uuid: "af2db7ab1bbc490f9bceae1317438337" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 38959 } } peers { permanent_uuid: "f151cc51113947f6a0ea2f463ed20681" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 37689 } }
I20260806 09:29:17.575747 29913 catalog_manager.cc:5495] T e6bcc8de9cdc453d8e0f2414e7e20d07 P af2db7ab1bbc490f9bceae1317438337 reported cstate change: term changed from 0 to 1, leader changed from <none> to af2db7ab1bbc490f9bceae1317438337 (127.20.113.2). New cstate: current_term: 1 leader_uuid: "af2db7ab1bbc490f9bceae1317438337" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb643e201d404a53a320740b57e36912" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 35129 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af2db7ab1bbc490f9bceae1317438337" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 38959 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f151cc51113947f6a0ea2f463ed20681" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 37689 } health_report { overall_health: UNKNOWN } } }
I20260806 09:29:17.635538 31903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:17.629344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44842 (local address 127.20.113.2:38959)
0806 09:29:17.630055 (+ 711us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:17.630059 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:17.630076 (+ 17us) server_negotiation.cc:407] Connection header received
0806 09:29:17.630168 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:17.630172 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:17.630250 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:17.630413 (+ 163us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 09:29:17.630797 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:17.631496 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:17.633132 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:17.633633 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:17.633780 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:17.634293 (+ 513us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 09:29:17.634903 (+ 610us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 09:29:17.635075 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:17.635143 (+ 68us) server_negotiation.cc:299] Negotiation successful
0806 09:29:17.635234 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":608,"thread_start_us":494,"threads_started":1}
I20260806 09:29:17.644160 31755 raft_consensus.cc:1261] T e6bcc8de9cdc453d8e0f2414e7e20d07 P fb643e201d404a53a320740b57e36912 [term 1 FOLLOWER]: Refusing update from remote peer af2db7ab1bbc490f9bceae1317438337: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 09:29:17.644160 31288 raft_consensus.cc:1261] T e6bcc8de9cdc453d8e0f2414e7e20d07 P f151cc51113947f6a0ea2f463ed20681 [term 1 FOLLOWER]: Refusing update from remote peer af2db7ab1bbc490f9bceae1317438337: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 09:29:17.645323 31886 consensus_queue.cc:1035] T e6bcc8de9cdc453d8e0f2414e7e20d07 P af2db7ab1bbc490f9bceae1317438337 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb643e201d404a53a320740b57e36912" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 35129 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 09:29:17.645766 31882 consensus_queue.cc:1035] T e6bcc8de9cdc453d8e0f2414e7e20d07 P af2db7ab1bbc490f9bceae1317438337 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f151cc51113947f6a0ea2f463ed20681" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 37689 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 09:29:17.676630 29913 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59218:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 09:29:17.676941 29913 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59218:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 09:29:17.685278 29913 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0cbb5d09bd23462caa29c64dd33fdd25: Sending DeleteTablet for 3 replicas of tablet e6bcc8de9cdc453d8e0f2414e7e20d07
I20260806 09:29:17.686904 31261 tablet_service.cc:1514] Processing DeleteTablet for tablet e6bcc8de9cdc453d8e0f2414e7e20d07 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:29:17 UTC) from {username='oryx', principal='oryx/127.20.113.62@KRBTEST.COM'} at 127.0.0.1:36242
I20260806 09:29:17.687045 31714 tablet_service.cc:1514] Processing DeleteTablet for tablet e6bcc8de9cdc453d8e0f2414e7e20d07 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:29:17 UTC) from {username='oryx', principal='oryx/127.20.113.62@KRBTEST.COM'} at 127.0.0.1:37894
I20260806 09:29:17.687628 31502 tablet_service.cc:1514] Processing DeleteTablet for tablet e6bcc8de9cdc453d8e0f2414e7e20d07 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:29:17 UTC) from {username='oryx', principal='oryx/127.20.113.62@KRBTEST.COM'} at 127.0.0.1:44834
I20260806 09:29:17.696506 31920 tablet_replica.cc:330] T e6bcc8de9cdc453d8e0f2414e7e20d07 P fb643e201d404a53a320740b57e36912: stopping tablet replica
I20260806 09:29:17.696853 31919 tablet_replica.cc:330] T e6bcc8de9cdc453d8e0f2414e7e20d07 P f151cc51113947f6a0ea2f463ed20681: stopping tablet replica
I20260806 09:29:17.697026 31920 raft_consensus.cc:2229] T e6bcc8de9cdc453d8e0f2414e7e20d07 P fb643e201d404a53a320740b57e36912 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 09:29:17.697270 31919 raft_consensus.cc:2229] T e6bcc8de9cdc453d8e0f2414e7e20d07 P f151cc51113947f6a0ea2f463ed20681 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 09:29:17.697585 31920 raft_consensus.cc:2258] T e6bcc8de9cdc453d8e0f2414e7e20d07 P fb643e201d404a53a320740b57e36912 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 09:29:17.697728 31919 raft_consensus.cc:2258] T e6bcc8de9cdc453d8e0f2414e7e20d07 P f151cc51113947f6a0ea2f463ed20681 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 09:29:17.698611 31923 tablet_replica.cc:330] T e6bcc8de9cdc453d8e0f2414e7e20d07 P af2db7ab1bbc490f9bceae1317438337: stopping tablet replica
I20260806 09:29:17.699019 31923 raft_consensus.cc:2229] T e6bcc8de9cdc453d8e0f2414e7e20d07 P af2db7ab1bbc490f9bceae1317438337 [term 1 LEADER]: Raft consensus shutting down.
I20260806 09:29:17.699740 31923 raft_consensus.cc:2258] T e6bcc8de9cdc453d8e0f2414e7e20d07 P af2db7ab1bbc490f9bceae1317438337 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 09:29:17.699899 31920 ts_tablet_manager.cc:1905] T e6bcc8de9cdc453d8e0f2414e7e20d07 P fb643e201d404a53a320740b57e36912: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 09:29:17.701666 31919 ts_tablet_manager.cc:1905] T e6bcc8de9cdc453d8e0f2414e7e20d07 P f151cc51113947f6a0ea2f463ed20681: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 09:29:17.708663 31920 ts_tablet_manager.cc:1918] T e6bcc8de9cdc453d8e0f2414e7e20d07 P fb643e201d404a53a320740b57e36912: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 09:29:17.708846 31920 log.cc:1196] T e6bcc8de9cdc453d8e0f2414e7e20d07 P fb643e201d404a53a320740b57e36912: Deleting WAL directory at /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/ts-2/wal/wals/e6bcc8de9cdc453d8e0f2414e7e20d07
I20260806 09:29:17.709033 31919 ts_tablet_manager.cc:1918] T e6bcc8de9cdc453d8e0f2414e7e20d07 P f151cc51113947f6a0ea2f463ed20681: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 09:29:17.709196 31919 log.cc:1196] T e6bcc8de9cdc453d8e0f2414e7e20d07 P f151cc51113947f6a0ea2f463ed20681: Deleting WAL directory at /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/ts-0/wal/wals/e6bcc8de9cdc453d8e0f2414e7e20d07
I20260806 09:29:17.709429 31920 ts_tablet_manager.cc:1939] T e6bcc8de9cdc453d8e0f2414e7e20d07 P fb643e201d404a53a320740b57e36912: Deleting consensus metadata
I20260806 09:29:17.709841 31919 ts_tablet_manager.cc:1939] T e6bcc8de9cdc453d8e0f2414e7e20d07 P f151cc51113947f6a0ea2f463ed20681: Deleting consensus metadata
I20260806 09:29:17.711676 29893 catalog_manager.cc:4841] TS f151cc51113947f6a0ea2f463ed20681 (127.20.113.1:37689): tablet e6bcc8de9cdc453d8e0f2414e7e20d07 (table test-table [id=f37e78b276324b6e9a42238136572c64]) successfully deleted
I20260806 09:29:17.711702 29891 catalog_manager.cc:4841] TS fb643e201d404a53a320740b57e36912 (127.20.113.3:35129): tablet e6bcc8de9cdc453d8e0f2414e7e20d07 (table test-table [id=f37e78b276324b6e9a42238136572c64]) successfully deleted
I20260806 09:29:17.713678 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 29869
I20260806 09:29:17.714195 31923 ts_tablet_manager.cc:1905] T e6bcc8de9cdc453d8e0f2414e7e20d07 P af2db7ab1bbc490f9bceae1317438337: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 09:29:17.714826 29963 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:29:17.727955 31923 ts_tablet_manager.cc:1918] T e6bcc8de9cdc453d8e0f2414e7e20d07 P af2db7ab1bbc490f9bceae1317438337: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 09:29:17.728160 31923 log.cc:1196] T e6bcc8de9cdc453d8e0f2414e7e20d07 P af2db7ab1bbc490f9bceae1317438337: Deleting WAL directory at /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/ts-1/wal/wals/e6bcc8de9cdc453d8e0f2414e7e20d07
I20260806 09:29:17.728822 31923 ts_tablet_manager.cc:1939] T e6bcc8de9cdc453d8e0f2414e7e20d07 P af2db7ab1bbc490f9bceae1317438337: Deleting consensus metadata
I20260806 09:29:17.742563 31925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:17.690077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43082 (local address 127.20.113.61:41373)
0806 09:29:17.712145 (+ 22068us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:17.712152 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:17.715779 (+ 3627us) server_negotiation.cc:407] Connection header received
0806 09:29:17.715866 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:17.715874 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:17.715992 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:17.716227 (+ 235us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:17.718565 (+ 2338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:17.719805 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:17.730732 (+ 10927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:17.731461 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:17.731607 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:17.735391 (+ 3784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:29:17.735400 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:29:17.735406 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:29:17.735452 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:29:17.739139 (+ 3687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:17.740131 (+ 992us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:17.740136 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:17.740140 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:17.740351 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:17.740784 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:17.740789 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:17.740793 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:17.741134 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:29:17.741400 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:17.741841 (+ 441us) server_negotiation.cc:299] Negotiation successful
0806 09:29:17.742087 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21874,"thread_start_us":22083,"threads_started":1}
I20260806 09:29:17.758821 31928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:17.712972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60308 (local address 127.20.113.60:42483)
0806 09:29:17.731113 (+ 18141us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:17.731120 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:17.742777 (+ 11657us) server_negotiation.cc:407] Connection header received
0806 09:29:17.742852 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:17.742857 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:17.742962 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:17.743068 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:17.745016 (+ 1948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:17.746201 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:17.748081 (+ 1880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:17.748816 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:17.748944 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:17.752215 (+ 3271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:29:17.752222 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:29:17.752227 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:29:17.752268 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:29:17.755815 (+ 3547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:17.756694 (+ 879us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:17.756699 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:17.756702 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:17.756854 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:17.757236 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:17.757241 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:17.757245 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:17.757581 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:29:17.757858 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:17.758181 (+ 323us) server_negotiation.cc:299] Negotiation successful
0806 09:29:17.758407 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17980,"thread_start_us":18149,"threads_started":1}
I20260806 09:29:17.843351 29894 catalog_manager.cc:4841] TS af2db7ab1bbc490f9bceae1317438337 (127.20.113.2:38959): tablet e6bcc8de9cdc453d8e0f2414e7e20d07 (table test-table [id=f37e78b276324b6e9a42238136572c64]) successfully deleted
I20260806 09:29:17.844106 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 29869
I20260806 09:29:17.901485 31925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:17.883705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43086 (local address 127.20.113.61:41373)
0806 09:29:17.883957 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:17.883968 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:17.885106 (+ 1138us) server_negotiation.cc:407] Connection header received
0806 09:29:17.885299 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:17.885306 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:17.885393 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:17.885503 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:17.888042 (+ 2539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:17.889047 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:17.890947 (+ 1900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:17.891566 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:17.891701 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:17.894681 (+ 2980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:29:17.894687 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:29:17.894694 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:29:17.894735 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:29:17.898210 (+ 3475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:17.899007 (+ 797us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:17.899013 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:17.899017 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:17.899158 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:17.899832 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:17.899838 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:17.899842 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:17.900137 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:29:17.900465 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:17.900807 (+ 342us) server_negotiation.cc:299] Negotiation successful
0806 09:29:17.901038 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260806 09:29:17.902335 31928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:17.885224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60314 (local address 127.20.113.60:42483)
0806 09:29:17.885462 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:17.885469 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:17.885991 (+ 522us) server_negotiation.cc:407] Connection header received
0806 09:29:17.886209 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:17.886215 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:17.886294 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:17.886398 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:17.888292 (+ 1894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:17.889420 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:17.891184 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:17.891673 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:17.891799 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:17.895408 (+ 3609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:29:17.895417 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:29:17.895424 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:29:17.895469 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:29:17.898422 (+ 2953us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:17.899263 (+ 841us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:17.899271 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:17.899274 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:17.899412 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:17.899921 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:17.899926 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:17.899930 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:17.900276 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:29:17.900556 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:17.901733 (+ 1177us) server_negotiation.cc:299] Negotiation successful
0806 09:29:17.901987 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260806 09:29:18.663461 31939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:18.662699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:45173 (local address 127.20.113.2:48587)
0806 09:29:18.663155 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:18.663290 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:45173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":220,"threads_started":1}
W20260806 09:29:18.663993 31608 heartbeater.cc:644] Failed to heartbeat to 127.20.113.62:45173 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.20.113.62:45173: connect: Connection refused (error 111)
I20260806 09:29:18.669955 31941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:18.668671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:45173 (local address 127.20.113.3:44847)
0806 09:29:18.669666 (+ 995us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:18.669793 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:45173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":889,"thread_start_us":836,"threads_started":1}
W20260806 09:29:18.670585 31821 heartbeater.cc:644] Failed to heartbeat to 127.20.113.62:45173 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.20.113.62:45173: connect: Connection refused (error 111)
I20260806 09:29:18.671471 31942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:18.669725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:45173 (local address 127.20.113.1:36233)
0806 09:29:18.671117 (+ 1392us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:18.671261 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:45173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1268,"thread_start_us":1161,"threads_started":1}
W20260806 09:29:18.672083 31372 heartbeater.cc:644] Failed to heartbeat to 127.20.113.62:45173 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.20.113.62:45173: connect: Connection refused (error 111)
I20260806 09:29:19.185407 31943 raft_consensus.cc:482] T 00000000000000000000000000000000 P e8ec8de97a86401e948b6ffb6a72e7d1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0cbb5d09bd23462caa29c64dd33fdd25)
I20260806 09:29:19.185675 31944 raft_consensus.cc:482] T 00000000000000000000000000000000 P 377f07a0afaa4800aa98d87714775150 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0cbb5d09bd23462caa29c64dd33fdd25)
I20260806 09:29:19.185626 31943 raft_consensus.cc:504] T 00000000000000000000000000000000 P e8ec8de97a86401e948b6ffb6a72e7d1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "0cbb5d09bd23462caa29c64dd33fdd25" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 45173 } } peers { permanent_uuid: "377f07a0afaa4800aa98d87714775150" member_type: VOTER last_known_addr { host: "127.20.113.61" port: 41373 } attrs { promote: false } } peers { permanent_uuid: "e8ec8de97a86401e948b6ffb6a72e7d1" member_type: VOTER last_known_addr { host: "127.20.113.60" port: 42483 } attrs { promote: false } }
I20260806 09:29:19.185933 31944 raft_consensus.cc:504] T 00000000000000000000000000000000 P 377f07a0afaa4800aa98d87714775150 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "0cbb5d09bd23462caa29c64dd33fdd25" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 45173 } } peers { permanent_uuid: "377f07a0afaa4800aa98d87714775150" member_type: VOTER last_known_addr { host: "127.20.113.61" port: 41373 } attrs { promote: false } } peers { permanent_uuid: "e8ec8de97a86401e948b6ffb6a72e7d1" member_type: VOTER last_known_addr { host: "127.20.113.60" port: 42483 } attrs { promote: false } }
I20260806 09:29:19.186580 31943 leader_election.cc:288] T 00000000000000000000000000000000 P e8ec8de97a86401e948b6ffb6a72e7d1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0cbb5d09bd23462caa29c64dd33fdd25 (127.20.113.62:45173), 377f07a0afaa4800aa98d87714775150 (127.20.113.61:41373)
I20260806 09:29:19.187070 31944 leader_election.cc:288] T 00000000000000000000000000000000 P 377f07a0afaa4800aa98d87714775150 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0cbb5d09bd23462caa29c64dd33fdd25 (127.20.113.62:45173), e8ec8de97a86401e948b6ffb6a72e7d1 (127.20.113.60:42483)
I20260806 09:29:19.188742 31950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:19.187147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:45173 (local address 127.0.0.1:59652)
0806 09:29:19.188111 (+ 964us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:19.188352 (+ 241us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:45173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":805,"thread_start_us":624,"threads_started":1}
W20260806 09:29:19.189108 30788 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.113.62:45173: connect: Connection refused (error 111)
I20260806 09:29:19.190973 31951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:19.190423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:45173 (local address 127.0.0.1:59664)
0806 09:29:19.190624 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:19.190773 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:45173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
W20260806 09:29:19.191223 30788 leader_election.cc:334] T 00000000000000000000000000000000 P 377f07a0afaa4800aa98d87714775150 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 0cbb5d09bd23462caa29c64dd33fdd25 (127.20.113.62:45173): Network error: Client connection negotiation failed: client connection to 127.20.113.62:45173: connect: Connection refused (error 111)
I20260806 09:29:19.192142 31945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:19.186728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:45173 (local address 127.0.0.1:59640)
0806 09:29:19.191592 (+ 4864us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:19.191785 (+ 193us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:45173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3491,"thread_start_us":3329,"threads_started":1}
W20260806 09:29:19.192467 31073 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.113.62:45173: connect: Connection refused (error 111)
I20260806 09:29:19.194195 31945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:19.193803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:45173 (local address 127.0.0.1:59666)
0806 09:29:19.193958 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:19.194060 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:45173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260806 09:29:19.194428 31073 leader_election.cc:334] T 00000000000000000000000000000000 P e8ec8de97a86401e948b6ffb6a72e7d1 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 0cbb5d09bd23462caa29c64dd33fdd25 (127.20.113.62:45173): Network error: Client connection negotiation failed: client connection to 127.20.113.62:45173: connect: Connection refused (error 111)
I20260806 09:29:19.199613 31950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:19.187555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.60:42483 (local address 127.0.0.1:60318)
0806 09:29:19.189025 (+ 1470us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:19.189046 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:19.189211 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:19.191113 (+ 1902us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:19.191117 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:19.191151 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:29:19.191486 (+ 335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:19.191497 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:19.194961 (+ 3464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:19.194966 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:19.197237 (+ 2271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:19.197255 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:19.198832 (+ 1577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:19.198838 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:19.199022 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:19.199056 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 09:29:19.199158 (+ 102us) client_negotiation.cc:240] Negotiation successful
0806 09:29:19.199291 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1347,"thread_start_us":2118,"threads_started":1}
I20260806 09:29:19.199637 31947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:19.186782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.61:41373 (local address 127.0.0.1:43094)
0806 09:29:19.190943 (+ 4161us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:19.190965 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:19.191203 (+ 238us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:19.191858 (+ 655us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:19.191863 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:19.191904 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:29:19.193509 (+ 1605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:19.193529 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:19.195086 (+ 1557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:19.195091 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:19.197174 (+ 2083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:19.197191 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:19.198836 (+ 1645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:19.198842 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:19.199028 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:19.199059 (+ 31us) client_negotiation.cc:769] Sending connection context
0806 09:29:19.199149 (+ 90us) client_negotiation.cc:240] Negotiation successful
0806 09:29:19.199280 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4050,"thread_start_us":5470,"threads_started":1}
I20260806 09:29:19.199744 31952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:19.187741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60318 (local address 127.20.113.60:42483)
0806 09:29:19.190751 (+ 3010us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:19.190762 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:19.190784 (+ 22us) server_negotiation.cc:407] Connection header received
0806 09:29:19.190864 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:19.190869 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:19.190959 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:19.191631 (+ 672us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:29:19.191879 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:19.194791 (+ 2912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:19.197411 (+ 2620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:19.198637 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:19.198773 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:19.198893 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:19.199210 (+ 317us) server_negotiation.cc:299] Negotiation successful
0806 09:29:19.199355 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2864,"thread_start_us":2719,"threads_started":1}
I20260806 09:29:19.199743 31949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:19.187040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43094 (local address 127.20.113.61:41373)
0806 09:29:19.187645 (+ 605us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:19.187652 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:19.191041 (+ 3389us) server_negotiation.cc:407] Connection header received
0806 09:29:19.191407 (+ 366us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:19.191412 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:19.191491 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:19.191570 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:29:19.193777 (+ 2207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:19.194955 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:19.197382 (+ 2427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:19.198629 (+ 1247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:19.198773 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:19.198910 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:19.199219 (+ 309us) server_negotiation.cc:299] Negotiation successful
0806 09:29:19.199362 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":472,"thread_start_us":267,"threads_started":1}
I20260806 09:29:19.200618 31107 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "377f07a0afaa4800aa98d87714775150" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "e8ec8de97a86401e948b6ffb6a72e7d1" is_pre_election: true
I20260806 09:29:19.200610 30827 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e8ec8de97a86401e948b6ffb6a72e7d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "377f07a0afaa4800aa98d87714775150" is_pre_election: true
I20260806 09:29:19.200928 30827 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 377f07a0afaa4800aa98d87714775150 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8ec8de97a86401e948b6ffb6a72e7d1 in term 1.
I20260806 09:29:19.200951 31107 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e8ec8de97a86401e948b6ffb6a72e7d1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 377f07a0afaa4800aa98d87714775150 in term 1.
I20260806 09:29:19.201426 31078 leader_election.cc:302] T 00000000000000000000000000000000 P e8ec8de97a86401e948b6ffb6a72e7d1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 377f07a0afaa4800aa98d87714775150, e8ec8de97a86401e948b6ffb6a72e7d1; no voters: 0cbb5d09bd23462caa29c64dd33fdd25
I20260806 09:29:19.201453 30794 leader_election.cc:302] T 00000000000000000000000000000000 P 377f07a0afaa4800aa98d87714775150 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 377f07a0afaa4800aa98d87714775150, e8ec8de97a86401e948b6ffb6a72e7d1; no voters: 0cbb5d09bd23462caa29c64dd33fdd25
I20260806 09:29:19.201805 31943 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e8ec8de97a86401e948b6ffb6a72e7d1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260806 09:29:19.201954 31944 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 377f07a0afaa4800aa98d87714775150 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260806 09:29:19.201967 31943 raft_consensus.cc:482] T 00000000000000000000000000000000 P e8ec8de97a86401e948b6ffb6a72e7d1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0cbb5d09bd23462caa29c64dd33fdd25)
I20260806 09:29:19.202097 31943 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e8ec8de97a86401e948b6ffb6a72e7d1 [term 1 FOLLOWER]: Advancing to term 2
I20260806 09:29:19.202098 31944 raft_consensus.cc:482] T 00000000000000000000000000000000 P 377f07a0afaa4800aa98d87714775150 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0cbb5d09bd23462caa29c64dd33fdd25)
I20260806 09:29:19.202209 31944 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 377f07a0afaa4800aa98d87714775150 [term 1 FOLLOWER]: Advancing to term 2
I20260806 09:29:19.204399 31944 raft_consensus.cc:504] T 00000000000000000000000000000000 P 377f07a0afaa4800aa98d87714775150 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "0cbb5d09bd23462caa29c64dd33fdd25" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 45173 } } peers { permanent_uuid: "377f07a0afaa4800aa98d87714775150" member_type: VOTER last_known_addr { host: "127.20.113.61" port: 41373 } attrs { promote: false } } peers { permanent_uuid: "e8ec8de97a86401e948b6ffb6a72e7d1" member_type: VOTER last_known_addr { host: "127.20.113.60" port: 42483 } attrs { promote: false } }
I20260806 09:29:19.204540 31943 raft_consensus.cc:504] T 00000000000000000000000000000000 P e8ec8de97a86401e948b6ffb6a72e7d1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "0cbb5d09bd23462caa29c64dd33fdd25" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 45173 } } peers { permanent_uuid: "377f07a0afaa4800aa98d87714775150" member_type: VOTER last_known_addr { host: "127.20.113.61" port: 41373 } attrs { promote: false } } peers { permanent_uuid: "e8ec8de97a86401e948b6ffb6a72e7d1" member_type: VOTER last_known_addr { host: "127.20.113.60" port: 42483 } attrs { promote: false } }
I20260806 09:29:19.205166 31944 leader_election.cc:288] T 00000000000000000000000000000000 P 377f07a0afaa4800aa98d87714775150 [CANDIDATE]: Term 2 election: Requested vote from peers 0cbb5d09bd23462caa29c64dd33fdd25 (127.20.113.62:45173), e8ec8de97a86401e948b6ffb6a72e7d1 (127.20.113.60:42483)
I20260806 09:29:19.205288 31943 leader_election.cc:288] T 00000000000000000000000000000000 P e8ec8de97a86401e948b6ffb6a72e7d1 [CANDIDATE]: Term 2 election: Requested vote from peers 0cbb5d09bd23462caa29c64dd33fdd25 (127.20.113.62:45173), 377f07a0afaa4800aa98d87714775150 (127.20.113.61:41373)
I20260806 09:29:19.205716 31107 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "377f07a0afaa4800aa98d87714775150" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "e8ec8de97a86401e948b6ffb6a72e7d1"
I20260806 09:29:19.205991 31947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:19.205574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:45173 (local address 127.0.0.1:59686)
0806 09:29:19.205733 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:19.205840 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:45173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260806 09:29:19.206053 31107 raft_consensus.cc:2379] T 00000000000000000000000000000000 P e8ec8de97a86401e948b6ffb6a72e7d1 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 377f07a0afaa4800aa98d87714775150 in current term 2: Already voted for candidate e8ec8de97a86401e948b6ffb6a72e7d1 in this term.
I20260806 09:29:19.206313 31950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:19.205236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:45173 (local address 127.0.0.1:59680)
0806 09:29:19.205829 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:19.205925 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:45173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":479}
I20260806 09:29:19.206341 30827 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e8ec8de97a86401e948b6ffb6a72e7d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "377f07a0afaa4800aa98d87714775150"
I20260806 09:29:19.206691 30827 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 377f07a0afaa4800aa98d87714775150 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate e8ec8de97a86401e948b6ffb6a72e7d1 in current term 2: Already voted for candidate 377f07a0afaa4800aa98d87714775150 in this term.
I20260806 09:29:19.207325 31951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:19.207007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:45173 (local address 127.0.0.1:59696)
0806 09:29:19.207156 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:19.207233 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:45173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260806 09:29:19.207545 30788 leader_election.cc:334] T 00000000000000000000000000000000 P 377f07a0afaa4800aa98d87714775150 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 0cbb5d09bd23462caa29c64dd33fdd25 (127.20.113.62:45173): Network error: Client connection negotiation failed: client connection to 127.20.113.62:45173: connect: Connection refused (error 111)
I20260806 09:29:19.207593 31947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:19.207230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:45173 (local address 127.0.0.1:59700)
0806 09:29:19.207382 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:19.207460 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:45173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260806 09:29:19.207669 30788 leader_election.cc:302] T 00000000000000000000000000000000 P 377f07a0afaa4800aa98d87714775150 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 377f07a0afaa4800aa98d87714775150; no voters: 0cbb5d09bd23462caa29c64dd33fdd25, e8ec8de97a86401e948b6ffb6a72e7d1
W20260806 09:29:19.207813 31073 leader_election.cc:334] T 00000000000000000000000000000000 P e8ec8de97a86401e948b6ffb6a72e7d1 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 0cbb5d09bd23462caa29c64dd33fdd25 (127.20.113.62:45173): Network error: Client connection negotiation failed: client connection to 127.20.113.62:45173: connect: Connection refused (error 111)
I20260806 09:29:19.208000 31944 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 377f07a0afaa4800aa98d87714775150 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260806 09:29:19.208197 31078 leader_election.cc:302] T 00000000000000000000000000000000 P e8ec8de97a86401e948b6ffb6a72e7d1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8ec8de97a86401e948b6ffb6a72e7d1; no voters: 0cbb5d09bd23462caa29c64dd33fdd25, 377f07a0afaa4800aa98d87714775150
I20260806 09:29:19.208487 31943 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e8ec8de97a86401e948b6ffb6a72e7d1 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260806 09:29:19.665657 31956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:19.664894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:45173 (local address 127.20.113.2:45285)
0806 09:29:19.665358 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:19.665461 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:45173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":233,"threads_started":1}
I20260806 09:29:19.672603 31958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:19.671418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:45173 (local address 127.20.113.3:54981)
0806 09:29:19.672126 (+ 708us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:19.672370 (+ 244us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:45173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":591,"thread_start_us":391,"threads_started":1}
I20260806 09:29:19.675082 31964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:19.673211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:45173 (local address 127.20.113.1:52783)
0806 09:29:19.674740 (+ 1529us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:19.674839 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:45173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1411,"thread_start_us":1315,"threads_started":1}
I20260806 09:29:20.667565 31965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:20.666717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:45173 (local address 127.20.113.2:60693)
0806 09:29:20.667318 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:20.667407 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:45173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":496,"thread_start_us":364,"threads_started":1}
I20260806 09:29:20.674793 31966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:20.673974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:45173 (local address 127.20.113.3:35115)
0806 09:29:20.674456 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:20.674592 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:45173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":546,"threads_started":1}
I20260806 09:29:20.676761 31967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:20.675908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:45173 (local address 127.20.113.1:52885)
0806 09:29:20.676480 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:20.676585 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:45173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":453,"threads_started":1}
I20260806 09:29:20.811111 31968 raft_consensus.cc:482] T 00000000000000000000000000000000 P 377f07a0afaa4800aa98d87714775150 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 09:29:20.811343 31968 raft_consensus.cc:504] T 00000000000000000000000000000000 P 377f07a0afaa4800aa98d87714775150 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "0cbb5d09bd23462caa29c64dd33fdd25" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 45173 } } peers { permanent_uuid: "377f07a0afaa4800aa98d87714775150" member_type: VOTER last_known_addr { host: "127.20.113.61" port: 41373 } attrs { promote: false } } peers { permanent_uuid: "e8ec8de97a86401e948b6ffb6a72e7d1" member_type: VOTER last_known_addr { host: "127.20.113.60" port: 42483 } attrs { promote: false } }
I20260806 09:29:20.812191 31968 leader_election.cc:288] T 00000000000000000000000000000000 P 377f07a0afaa4800aa98d87714775150 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0cbb5d09bd23462caa29c64dd33fdd25 (127.20.113.62:45173), e8ec8de97a86401e948b6ffb6a72e7d1 (127.20.113.60:42483)
I20260806 09:29:20.812851 31107 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "377f07a0afaa4800aa98d87714775150" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "e8ec8de97a86401e948b6ffb6a72e7d1" is_pre_election: true
I20260806 09:29:20.813163 31107 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e8ec8de97a86401e948b6ffb6a72e7d1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 377f07a0afaa4800aa98d87714775150 in term 2.
I20260806 09:29:20.813452 31969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:20.812375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:45173 (local address 127.0.0.1:59956)
0806 09:29:20.813075 (+ 700us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:20.813207 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:45173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":562,"thread_start_us":417,"threads_started":1}
I20260806 09:29:20.813733 30794 leader_election.cc:302] T 00000000000000000000000000000000 P 377f07a0afaa4800aa98d87714775150 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 377f07a0afaa4800aa98d87714775150, e8ec8de97a86401e948b6ffb6a72e7d1; no voters:
I20260806 09:29:20.814088 31968 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 377f07a0afaa4800aa98d87714775150 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260806 09:29:20.814222 31968 raft_consensus.cc:482] T 00000000000000000000000000000000 P 377f07a0afaa4800aa98d87714775150 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 09:29:20.814355 31968 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 377f07a0afaa4800aa98d87714775150 [term 2 FOLLOWER]: Advancing to term 3
I20260806 09:29:20.815583 31969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:20.815109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:45173 (local address 127.0.0.1:59966)
0806 09:29:20.815318 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:20.815403 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:45173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
W20260806 09:29:20.815786 30788 leader_election.cc:334] T 00000000000000000000000000000000 P 377f07a0afaa4800aa98d87714775150 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 0cbb5d09bd23462caa29c64dd33fdd25 (127.20.113.62:45173): Network error: Client connection negotiation failed: client connection to 127.20.113.62:45173: connect: Connection refused (error 111)
I20260806 09:29:20.817541 31968 raft_consensus.cc:504] T 00000000000000000000000000000000 P 377f07a0afaa4800aa98d87714775150 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "0cbb5d09bd23462caa29c64dd33fdd25" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 45173 } } peers { permanent_uuid: "377f07a0afaa4800aa98d87714775150" member_type: VOTER last_known_addr { host: "127.20.113.61" port: 41373 } attrs { promote: false } } peers { permanent_uuid: "e8ec8de97a86401e948b6ffb6a72e7d1" member_type: VOTER last_known_addr { host: "127.20.113.60" port: 42483 } attrs { promote: false } }
I20260806 09:29:20.818246 31968 leader_election.cc:288] T 00000000000000000000000000000000 P 377f07a0afaa4800aa98d87714775150 [CANDIDATE]: Term 3 election: Requested vote from peers 0cbb5d09bd23462caa29c64dd33fdd25 (127.20.113.62:45173), e8ec8de97a86401e948b6ffb6a72e7d1 (127.20.113.60:42483)
I20260806 09:29:20.818856 31969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:20.818369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:45173 (local address 127.0.0.1:59974)
0806 09:29:20.818560 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:20.818672 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:45173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122}
I20260806 09:29:20.818799 31107 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "377f07a0afaa4800aa98d87714775150" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "e8ec8de97a86401e948b6ffb6a72e7d1"
I20260806 09:29:20.819036 31107 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e8ec8de97a86401e948b6ffb6a72e7d1 [term 2 FOLLOWER]: Advancing to term 3
I20260806 09:29:20.819906 31969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:20.819499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:45173 (local address 127.0.0.1:59986)
0806 09:29:20.819662 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:20.819761 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:45173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260806 09:29:20.820065 30788 leader_election.cc:334] T 00000000000000000000000000000000 P 377f07a0afaa4800aa98d87714775150 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 0cbb5d09bd23462caa29c64dd33fdd25 (127.20.113.62:45173): Network error: Client connection negotiation failed: client connection to 127.20.113.62:45173: connect: Connection refused (error 111)
I20260806 09:29:20.822482 31107 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e8ec8de97a86401e948b6ffb6a72e7d1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 377f07a0afaa4800aa98d87714775150 in term 3.
I20260806 09:29:20.822996 30794 leader_election.cc:302] T 00000000000000000000000000000000 P 377f07a0afaa4800aa98d87714775150 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 377f07a0afaa4800aa98d87714775150, e8ec8de97a86401e948b6ffb6a72e7d1; no voters: 0cbb5d09bd23462caa29c64dd33fdd25
I20260806 09:29:20.823314 31968 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 377f07a0afaa4800aa98d87714775150 [term 3 FOLLOWER]: Leader election won for term 3
I20260806 09:29:20.823817 31968 raft_consensus.cc:686] T 00000000000000000000000000000000 P 377f07a0afaa4800aa98d87714775150 [term 3 LEADER]: Becoming Leader. State: Replica: 377f07a0afaa4800aa98d87714775150, State: Running, Role: LEADER
I20260806 09:29:20.824342 31968 consensus_queue.cc:237] T 00000000000000000000000000000000 P 377f07a0afaa4800aa98d87714775150 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "0cbb5d09bd23462caa29c64dd33fdd25" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 45173 } } peers { permanent_uuid: "377f07a0afaa4800aa98d87714775150" member_type: VOTER last_known_addr { host: "127.20.113.61" port: 41373 } attrs { promote: false } } peers { permanent_uuid: "e8ec8de97a86401e948b6ffb6a72e7d1" member_type: VOTER last_known_addr { host: "127.20.113.60" port: 42483 } attrs { promote: false } }
I20260806 09:29:20.826830 31971 sys_catalog.cc:455] T 00000000000000000000000000000000 P 377f07a0afaa4800aa98d87714775150 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 377f07a0afaa4800aa98d87714775150. Latest consensus state: current_term: 3 leader_uuid: "377f07a0afaa4800aa98d87714775150" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "0cbb5d09bd23462caa29c64dd33fdd25" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 45173 } } peers { permanent_uuid: "377f07a0afaa4800aa98d87714775150" member_type: VOTER last_known_addr { host: "127.20.113.61" port: 41373 } attrs { promote: false } } peers { permanent_uuid: "e8ec8de97a86401e948b6ffb6a72e7d1" member_type: VOTER last_known_addr { host: "127.20.113.60" port: 42483 } attrs { promote: false } } }
I20260806 09:29:20.827189 31971 sys_catalog.cc:458] T 00000000000000000000000000000000 P 377f07a0afaa4800aa98d87714775150 [sys.catalog]: This master's current role is: LEADER
I20260806 09:29:20.829994 31975 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 09:29:20.834645 31975 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 09:29:20.835695 31975 catalog_manager.cc:1259] Loaded cluster ID: e50f8b022b59446eaf1c3aaefb8eb3e7
I20260806 09:29:20.835764 31975 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 09:29:20.837446 31975 catalog_manager.cc:1504] Loading token signing keys...
I20260806 09:29:20.838552 31975 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 377f07a0afaa4800aa98d87714775150: Loaded TSK: 0
I20260806 09:29:20.839566 31975 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 09:29:20.855863 30812 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43086:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 09:29:20.863065 30812 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 09:29:20.867717 31969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:20.867164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:45173 (local address 127.0.0.1:60002)
0806 09:29:20.867419 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:20.867525 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:45173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124}
I20260806 09:29:20.867700 31107 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e8ec8de97a86401e948b6ffb6a72e7d1 [term 3 FOLLOWER]: Refusing update from remote peer 377f07a0afaa4800aa98d87714775150: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260806 09:29:20.868568 31971 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 377f07a0afaa4800aa98d87714775150 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8ec8de97a86401e948b6ffb6a72e7d1" member_type: VOTER last_known_addr { host: "127.20.113.60" port: 42483 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260806 09:29:20.868875 31969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:20.868515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:45173 (local address 127.0.0.1:60006)
0806 09:29:20.868670 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:20.868737 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:45173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260806 09:29:20.869112 30788 consensus_peers.cc:487] T 00000000000000000000000000000000 P 377f07a0afaa4800aa98d87714775150 -> Peer 0cbb5d09bd23462caa29c64dd33fdd25 (127.20.113.62:45173): Couldn't send request to peer 0cbb5d09bd23462caa29c64dd33fdd25. Status: Network error: Client connection negotiation failed: client connection to 127.20.113.62:45173: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260806 09:29:20.871827 31977 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8ec8de97a86401e948b6ffb6a72e7d1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 377f07a0afaa4800aa98d87714775150. Latest consensus state: current_term: 3 leader_uuid: "377f07a0afaa4800aa98d87714775150" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "0cbb5d09bd23462caa29c64dd33fdd25" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 45173 } } peers { permanent_uuid: "377f07a0afaa4800aa98d87714775150" member_type: VOTER last_known_addr { host: "127.20.113.61" port: 41373 } attrs { promote: false } } peers { permanent_uuid: "e8ec8de97a86401e948b6ffb6a72e7d1" member_type: VOTER last_known_addr { host: "127.20.113.60" port: 42483 } attrs { promote: false } } }
I20260806 09:29:20.872202 31977 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8ec8de97a86401e948b6ffb6a72e7d1 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 09:29:20.873208 31971 sys_catalog.cc:455] T 00000000000000000000000000000000 P 377f07a0afaa4800aa98d87714775150 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "377f07a0afaa4800aa98d87714775150" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "0cbb5d09bd23462caa29c64dd33fdd25" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 45173 } } peers { permanent_uuid: "377f07a0afaa4800aa98d87714775150" member_type: VOTER last_known_addr { host: "127.20.113.61" port: 41373 } attrs { promote: false } } peers { permanent_uuid: "e8ec8de97a86401e948b6ffb6a72e7d1" member_type: VOTER last_known_addr { host: "127.20.113.60" port: 42483 } attrs { promote: false } } }
I20260806 09:29:20.873530 31971 sys_catalog.cc:458] T 00000000000000000000000000000000 P 377f07a0afaa4800aa98d87714775150 [sys.catalog]: This master's current role is: LEADER
I20260806 09:29:20.874152 31977 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8ec8de97a86401e948b6ffb6a72e7d1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "377f07a0afaa4800aa98d87714775150" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "0cbb5d09bd23462caa29c64dd33fdd25" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 45173 } } peers { permanent_uuid: "377f07a0afaa4800aa98d87714775150" member_type: VOTER last_known_addr { host: "127.20.113.61" port: 41373 } attrs { promote: false } } peers { permanent_uuid: "e8ec8de97a86401e948b6ffb6a72e7d1" member_type: VOTER last_known_addr { host: "127.20.113.60" port: 42483 } attrs { promote: false } } }
I20260806 09:29:20.874388 31977 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8ec8de97a86401e948b6ffb6a72e7d1 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 09:29:20.893149 31969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:20.882229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.2:38959 (local address 127.0.0.1:44852)
0806 09:29:20.882376 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:20.882402 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:20.882546 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:20.885829 (+ 3283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:20.885834 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:20.885860 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:29:20.886141 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:20.886149 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:20.888262 (+ 2113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:20.888268 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:20.890827 (+ 2559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:20.890838 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:20.892466 (+ 1628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:20.892472 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:20.892654 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:20.892688 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 09:29:20.892742 (+ 54us) client_negotiation.cc:240] Negotiation successful
0806 09:29:20.892852 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32,"mutex_wait_us":479}
I20260806 09:29:20.893195 31984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:20.882571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.3:35129 (local address 127.0.0.1:37908)
0806 09:29:20.883206 (+ 635us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:20.883226 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:20.883336 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:20.885109 (+ 1773us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:20.885114 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:20.885141 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:29:20.885581 (+ 440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:20.885593 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:20.887749 (+ 2156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:20.887754 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:20.890147 (+ 2393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:20.890161 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:20.892180 (+ 2019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:20.892186 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:20.892459 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:20.892499 (+ 40us) client_negotiation.cc:769] Sending connection context
0806 09:29:20.892595 (+ 96us) client_negotiation.cc:240] Negotiation successful
0806 09:29:20.892752 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":470,"mutex_wait_us":2,"thread_start_us":1510,"threads_started":1}
I20260806 09:29:20.893705 31985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:20.882715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37908 (local address 127.20.113.3:35129)
0806 09:29:20.884757 (+ 2042us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:20.884761 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:20.884777 (+ 16us) server_negotiation.cc:407] Connection header received
0806 09:29:20.884844 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:20.884847 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:20.884945 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:20.885770 (+ 825us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:29:20.886065 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:20.887587 (+ 1522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:20.890308 (+ 2721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:20.891978 (+ 1670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:20.892136 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:20.892285 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:20.893164 (+ 879us) server_negotiation.cc:299] Negotiation successful
0806 09:29:20.893327 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1907,"thread_start_us":1771,"threads_started":1}
I20260806 09:29:20.894562 31714 tablet_service.cc:1467] Processing CreateTablet for tablet 30e2edc028634ee8b972bb1ed0fcab1b (DEFAULT_TABLE table=test-table [id=88d14ccbd13040f6bf14d0e3ec362b4e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:29:20.894744 31983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:20.882345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44852 (local address 127.20.113.2:38959)
0806 09:29:20.885552 (+ 3207us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:20.885557 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:20.885572 (+ 15us) server_negotiation.cc:407] Connection header received
0806 09:29:20.885630 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:20.885634 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:20.885689 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:20.885812 (+ 123us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:29:20.887059 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:20.888064 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:20.891207 (+ 3143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:20.892184 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:20.894066 (+ 1882us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:20.894175 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:20.894228 (+ 53us) server_negotiation.cc:299] Negotiation successful
0806 09:29:20.894395 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3035,"thread_start_us":2932,"threads_started":1}
I20260806 09:29:20.895059 31714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30e2edc028634ee8b972bb1ed0fcab1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:29:20.895416 31502 tablet_service.cc:1467] Processing CreateTablet for tablet 30e2edc028634ee8b972bb1ed0fcab1b (DEFAULT_TABLE table=test-table [id=88d14ccbd13040f6bf14d0e3ec362b4e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:29:20.895876 31502 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30e2edc028634ee8b972bb1ed0fcab1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:29:20.900290 31988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:20.885165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.1:37689 (local address 127.0.0.1:36254)
0806 09:29:20.892804 (+ 7639us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:20.892818 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:20.892964 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:20.893960 (+ 996us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:20.893963 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:20.893983 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:29:20.894277 (+ 294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:20.894289 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:20.895638 (+ 1349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:20.895643 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:20.897733 (+ 2090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:20.897750 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:20.899580 (+ 1830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:20.899586 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:20.899746 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:20.899776 (+ 30us) client_negotiation.cc:769] Sending connection context
0806 09:29:20.899856 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 09:29:20.899974 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7519,"thread_start_us":7384,"threads_started":1}
I20260806 09:29:20.901077 31989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:20.885580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36254 (local address 127.20.113.1:37689)
0806 09:29:20.886720 (+ 1140us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:20.886727 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:20.892920 (+ 6193us) server_negotiation.cc:407] Connection header received
0806 09:29:20.893116 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:20.893122 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:20.893217 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:20.893455 (+ 238us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:29:20.894491 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:20.895486 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:20.897921 (+ 2435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:20.899126 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:20.899272 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:20.899378 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:20.900608 (+ 1230us) server_negotiation.cc:299] Negotiation successful
0806 09:29:20.900753 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":943,"thread_start_us":788,"threads_started":1}
I20260806 09:29:20.901095 31990 tablet_bootstrap.cc:492] T 30e2edc028634ee8b972bb1ed0fcab1b P fb643e201d404a53a320740b57e36912: Bootstrap starting.
I20260806 09:29:20.902048 31991 tablet_bootstrap.cc:492] T 30e2edc028634ee8b972bb1ed0fcab1b P af2db7ab1bbc490f9bceae1317438337: Bootstrap starting.
I20260806 09:29:20.902148 31261 tablet_service.cc:1467] Processing CreateTablet for tablet 30e2edc028634ee8b972bb1ed0fcab1b (DEFAULT_TABLE table=test-table [id=88d14ccbd13040f6bf14d0e3ec362b4e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:29:20.902593 31261 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30e2edc028634ee8b972bb1ed0fcab1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:29:20.902992 31990 tablet_bootstrap.cc:654] T 30e2edc028634ee8b972bb1ed0fcab1b P fb643e201d404a53a320740b57e36912: Neither blocks nor log segments found. Creating new log.
I20260806 09:29:20.903570 31991 tablet_bootstrap.cc:654] T 30e2edc028634ee8b972bb1ed0fcab1b P af2db7ab1bbc490f9bceae1317438337: Neither blocks nor log segments found. Creating new log.
I20260806 09:29:20.905544 31990 tablet_bootstrap.cc:492] T 30e2edc028634ee8b972bb1ed0fcab1b P fb643e201d404a53a320740b57e36912: No bootstrap required, opened a new log
I20260806 09:29:20.905777 31990 ts_tablet_manager.cc:1397] T 30e2edc028634ee8b972bb1ed0fcab1b P fb643e201d404a53a320740b57e36912: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 09:29:20.906252 31991 tablet_bootstrap.cc:492] T 30e2edc028634ee8b972bb1ed0fcab1b P af2db7ab1bbc490f9bceae1317438337: No bootstrap required, opened a new log
I20260806 09:29:20.906416 31991 ts_tablet_manager.cc:1397] T 30e2edc028634ee8b972bb1ed0fcab1b P af2db7ab1bbc490f9bceae1317438337: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260806 09:29:20.906725 31990 raft_consensus.cc:348] T 30e2edc028634ee8b972bb1ed0fcab1b P fb643e201d404a53a320740b57e36912 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af2db7ab1bbc490f9bceae1317438337" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 38959 } } peers { permanent_uuid: "fb643e201d404a53a320740b57e36912" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 35129 } } peers { permanent_uuid: "f151cc51113947f6a0ea2f463ed20681" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 37689 } }
I20260806 09:29:20.906978 31990 raft_consensus.cc:374] T 30e2edc028634ee8b972bb1ed0fcab1b P fb643e201d404a53a320740b57e36912 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:29:20.907080 31990 raft_consensus.cc:729] T 30e2edc028634ee8b972bb1ed0fcab1b P fb643e201d404a53a320740b57e36912 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb643e201d404a53a320740b57e36912, State: Initialized, Role: FOLLOWER
I20260806 09:29:20.907178 31991 raft_consensus.cc:348] T 30e2edc028634ee8b972bb1ed0fcab1b P af2db7ab1bbc490f9bceae1317438337 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af2db7ab1bbc490f9bceae1317438337" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 38959 } } peers { permanent_uuid: "fb643e201d404a53a320740b57e36912" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 35129 } } peers { permanent_uuid: "f151cc51113947f6a0ea2f463ed20681" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 37689 } }
I20260806 09:29:20.907461 31991 raft_consensus.cc:374] T 30e2edc028634ee8b972bb1ed0fcab1b P af2db7ab1bbc490f9bceae1317438337 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:29:20.907380 31990 consensus_queue.cc:260] T 30e2edc028634ee8b972bb1ed0fcab1b P fb643e201d404a53a320740b57e36912 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af2db7ab1bbc490f9bceae1317438337" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 38959 } } peers { permanent_uuid: "fb643e201d404a53a320740b57e36912" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 35129 } } peers { permanent_uuid: "f151cc51113947f6a0ea2f463ed20681" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 37689 } }
I20260806 09:29:20.907558 31991 raft_consensus.cc:729] T 30e2edc028634ee8b972bb1ed0fcab1b P af2db7ab1bbc490f9bceae1317438337 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af2db7ab1bbc490f9bceae1317438337, State: Initialized, Role: FOLLOWER
I20260806 09:29:20.907759 31991 consensus_queue.cc:260] T 30e2edc028634ee8b972bb1ed0fcab1b P af2db7ab1bbc490f9bceae1317438337 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af2db7ab1bbc490f9bceae1317438337" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 38959 } } peers { permanent_uuid: "fb643e201d404a53a320740b57e36912" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 35129 } } peers { permanent_uuid: "f151cc51113947f6a0ea2f463ed20681" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 37689 } }
I20260806 09:29:20.908365 31990 ts_tablet_manager.cc:1428] T 30e2edc028634ee8b972bb1ed0fcab1b P fb643e201d404a53a320740b57e36912: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 09:29:20.909623 31966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:20.909016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:45173 (local address 127.20.113.3:57105)
0806 09:29:20.909316 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:20.909439 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:45173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137}
I20260806 09:29:20.911945 31822 heartbeater.cc:503] Master 127.20.113.61:41373 was elected leader, sending a full tablet report...
I20260806 09:29:20.912588 31991 ts_tablet_manager.cc:1428] T 30e2edc028634ee8b972bb1ed0fcab1b P af2db7ab1bbc490f9bceae1317438337: Time spent starting tablet: real 0.006s user 0.002s sys 0.005s
I20260806 09:29:20.912624 31966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:20.912084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:45173 (local address 127.20.113.3:51091)
0806 09:29:20.912375 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:20.912455 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:45173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181}
I20260806 09:29:20.913717 31965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:20.913238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:45173 (local address 127.20.113.2:44723)
0806 09:29:20.913484 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:20.913578 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:45173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124}
I20260806 09:29:20.914237 31997 tablet_bootstrap.cc:492] T 30e2edc028634ee8b972bb1ed0fcab1b P f151cc51113947f6a0ea2f463ed20681: Bootstrap starting.
I20260806 09:29:20.915726 31609 heartbeater.cc:503] Master 127.20.113.61:41373 was elected leader, sending a full tablet report...
I20260806 09:29:20.916244 31997 tablet_bootstrap.cc:654] T 30e2edc028634ee8b972bb1ed0fcab1b P f151cc51113947f6a0ea2f463ed20681: Neither blocks nor log segments found. Creating new log.
I20260806 09:29:20.916394 31965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:20.915849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:45173 (local address 127.20.113.2:47145)
0806 09:29:20.916111 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:20.916196 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:45173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150}
I20260806 09:29:20.918711 31997 tablet_bootstrap.cc:492] T 30e2edc028634ee8b972bb1ed0fcab1b P f151cc51113947f6a0ea2f463ed20681: No bootstrap required, opened a new log
I20260806 09:29:20.918885 31997 ts_tablet_manager.cc:1397] T 30e2edc028634ee8b972bb1ed0fcab1b P f151cc51113947f6a0ea2f463ed20681: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260806 09:29:20.919513 31997 raft_consensus.cc:348] T 30e2edc028634ee8b972bb1ed0fcab1b P f151cc51113947f6a0ea2f463ed20681 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af2db7ab1bbc490f9bceae1317438337" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 38959 } } peers { permanent_uuid: "fb643e201d404a53a320740b57e36912" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 35129 } } peers { permanent_uuid: "f151cc51113947f6a0ea2f463ed20681" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 37689 } }
I20260806 09:29:20.919716 31997 raft_consensus.cc:374] T 30e2edc028634ee8b972bb1ed0fcab1b P f151cc51113947f6a0ea2f463ed20681 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:29:20.919811 31997 raft_consensus.cc:729] T 30e2edc028634ee8b972bb1ed0fcab1b P f151cc51113947f6a0ea2f463ed20681 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f151cc51113947f6a0ea2f463ed20681, State: Initialized, Role: FOLLOWER
I20260806 09:29:20.920059 31997 consensus_queue.cc:260] T 30e2edc028634ee8b972bb1ed0fcab1b P f151cc51113947f6a0ea2f463ed20681 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af2db7ab1bbc490f9bceae1317438337" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 38959 } } peers { permanent_uuid: "fb643e201d404a53a320740b57e36912" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 35129 } } peers { permanent_uuid: "f151cc51113947f6a0ea2f463ed20681" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 37689 } }
I20260806 09:29:20.920863 31997 ts_tablet_manager.cc:1428] T 30e2edc028634ee8b972bb1ed0fcab1b P f151cc51113947f6a0ea2f463ed20681: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 09:29:20.921932 31967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:20.921461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:45173 (local address 127.20.113.1:57289)
0806 09:29:20.921709 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:20.921801 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:45173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161}
I20260806 09:29:20.923807 31373 heartbeater.cc:503] Master 127.20.113.61:41373 was elected leader, sending a full tablet report...
I20260806 09:29:20.923919 31967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:20.923472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:45173 (local address 127.20.113.1:40949)
0806 09:29:20.923681 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:20.923756 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:45173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117}
I20260806 09:29:21.122161 32000 raft_consensus.cc:482] T 30e2edc028634ee8b972bb1ed0fcab1b P f151cc51113947f6a0ea2f463ed20681 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 09:29:21.122437 32000 raft_consensus.cc:504] T 30e2edc028634ee8b972bb1ed0fcab1b P f151cc51113947f6a0ea2f463ed20681 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af2db7ab1bbc490f9bceae1317438337" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 38959 } } peers { permanent_uuid: "fb643e201d404a53a320740b57e36912" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 35129 } } peers { permanent_uuid: "f151cc51113947f6a0ea2f463ed20681" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 37689 } }
I20260806 09:29:21.124171 32000 leader_election.cc:288] T 30e2edc028634ee8b972bb1ed0fcab1b P f151cc51113947f6a0ea2f463ed20681 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers af2db7ab1bbc490f9bceae1317438337 (127.20.113.2:38959), fb643e201d404a53a320740b57e36912 (127.20.113.3:35129)
I20260806 09:29:21.132608 31985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:21.124768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.1:37717 (local address 127.20.113.3:35129)
0806 09:29:21.124971 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:21.124979 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:21.125652 (+ 673us) server_negotiation.cc:407] Connection header received
0806 09:29:21.125853 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:21.125860 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:21.125942 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:21.126009 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:29:21.126668 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:21.127340 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:21.130211 (+ 2871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:21.131681 (+ 1470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:21.131851 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:21.131960 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:21.132136 (+ 176us) server_negotiation.cc:299] Negotiation successful
0806 09:29:21.132262 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 09:29:21.132668 32002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:21.124540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.3:35129 (local address 127.20.113.1:37717)
0806 09:29:21.125515 (+ 975us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:21.125552 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:21.125682 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:21.126207 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:21.126214 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:21.126240 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:29:21.126509 (+ 269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:21.126517 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:21.127476 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:21.127481 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:21.130036 (+ 2555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:21.130051 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:21.131858 (+ 1807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:21.131862 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:21.131979 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:21.132001 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 09:29:21.132102 (+ 101us) client_negotiation.cc:240] Negotiation successful
0806 09:29:21.132214 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":849,"mutex_wait_us":158,"thread_start_us":560,"threads_started":1}
I20260806 09:29:21.134377 31755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30e2edc028634ee8b972bb1ed0fcab1b" candidate_uuid: "f151cc51113947f6a0ea2f463ed20681" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb643e201d404a53a320740b57e36912" is_pre_election: true
I20260806 09:29:21.134616 31983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:21.124706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.1:43129 (local address 127.20.113.2:38959)
0806 09:29:21.124911 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:21.124918 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:21.125395 (+ 477us) server_negotiation.cc:407] Connection header received
0806 09:29:21.125496 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:21.125505 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:21.125614 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:21.125718 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:29:21.126301 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:21.127466 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:21.132049 (+ 4583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:21.133406 (+ 1357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:21.133605 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:21.133692 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:21.134162 (+ 470us) server_negotiation.cc:299] Negotiation successful
0806 09:29:21.134330 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 09:29:21.134635 31755 raft_consensus.cc:2454] T 30e2edc028634ee8b972bb1ed0fcab1b P fb643e201d404a53a320740b57e36912 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f151cc51113947f6a0ea2f463ed20681 in term 0.
I20260806 09:29:21.135267 31967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:21.124408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.2:38959 (local address 127.20.113.1:43129)
0806 09:29:21.125166 (+ 758us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:21.125189 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:21.125348 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:21.125751 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:21.125757 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:21.125789 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:29:21.126120 (+ 331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:21.126136 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:21.127608 (+ 1472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:21.127613 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:21.130038 (+ 2425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:21.130051 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:21.133745 (+ 3694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:21.133756 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:21.133972 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:21.134005 (+ 33us) client_negotiation.cc:769] Sending connection context
0806 09:29:21.134112 (+ 107us) client_negotiation.cc:240] Negotiation successful
0806 09:29:21.135027 (+ 915us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":601,"mutex_wait_us":153}
I20260806 09:29:21.135258 31193 leader_election.cc:302] T 30e2edc028634ee8b972bb1ed0fcab1b P f151cc51113947f6a0ea2f463ed20681 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f151cc51113947f6a0ea2f463ed20681, fb643e201d404a53a320740b57e36912; no voters:
I20260806 09:29:21.135876 31996 raft_consensus.cc:482] T 30e2edc028634ee8b972bb1ed0fcab1b P af2db7ab1bbc490f9bceae1317438337 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 09:29:21.136092 32000 raft_consensus.cc:2781] T 30e2edc028634ee8b972bb1ed0fcab1b P f151cc51113947f6a0ea2f463ed20681 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 09:29:21.136225 32000 raft_consensus.cc:482] T 30e2edc028634ee8b972bb1ed0fcab1b P f151cc51113947f6a0ea2f463ed20681 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 09:29:21.136349 32000 raft_consensus.cc:3037] T 30e2edc028634ee8b972bb1ed0fcab1b P f151cc51113947f6a0ea2f463ed20681 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:29:21.136125 31996 raft_consensus.cc:504] T 30e2edc028634ee8b972bb1ed0fcab1b P af2db7ab1bbc490f9bceae1317438337 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af2db7ab1bbc490f9bceae1317438337" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 38959 } } peers { permanent_uuid: "fb643e201d404a53a320740b57e36912" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 35129 } } peers { permanent_uuid: "f151cc51113947f6a0ea2f463ed20681" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 37689 } }
I20260806 09:29:21.136423 31528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30e2edc028634ee8b972bb1ed0fcab1b" candidate_uuid: "f151cc51113947f6a0ea2f463ed20681" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af2db7ab1bbc490f9bceae1317438337" is_pre_election: true
I20260806 09:29:21.136875 31528 raft_consensus.cc:2454] T 30e2edc028634ee8b972bb1ed0fcab1b P af2db7ab1bbc490f9bceae1317438337 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f151cc51113947f6a0ea2f463ed20681 in term 0.
I20260806 09:29:21.137217 31996 leader_election.cc:288] T 30e2edc028634ee8b972bb1ed0fcab1b P af2db7ab1bbc490f9bceae1317438337 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fb643e201d404a53a320740b57e36912 (127.20.113.3:35129), f151cc51113947f6a0ea2f463ed20681 (127.20.113.1:37689)
I20260806 09:29:21.140118 32000 raft_consensus.cc:504] T 30e2edc028634ee8b972bb1ed0fcab1b P f151cc51113947f6a0ea2f463ed20681 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af2db7ab1bbc490f9bceae1317438337" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 38959 } } peers { permanent_uuid: "fb643e201d404a53a320740b57e36912" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 35129 } } peers { permanent_uuid: "f151cc51113947f6a0ea2f463ed20681" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 37689 } }
I20260806 09:29:21.140969 31288 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30e2edc028634ee8b972bb1ed0fcab1b" candidate_uuid: "af2db7ab1bbc490f9bceae1317438337" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f151cc51113947f6a0ea2f463ed20681" is_pre_election: true
I20260806 09:29:21.140990 31755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30e2edc028634ee8b972bb1ed0fcab1b" candidate_uuid: "af2db7ab1bbc490f9bceae1317438337" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb643e201d404a53a320740b57e36912" is_pre_election: true
I20260806 09:29:21.141253 31755 raft_consensus.cc:2454] T 30e2edc028634ee8b972bb1ed0fcab1b P fb643e201d404a53a320740b57e36912 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate af2db7ab1bbc490f9bceae1317438337 in term 0.
I20260806 09:29:21.141295 31288 raft_consensus.cc:2379] T 30e2edc028634ee8b972bb1ed0fcab1b P f151cc51113947f6a0ea2f463ed20681 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate af2db7ab1bbc490f9bceae1317438337 in current term 1: Already voted for candidate f151cc51113947f6a0ea2f463ed20681 in this term.
I20260806 09:29:21.141647 31429 leader_election.cc:302] T 30e2edc028634ee8b972bb1ed0fcab1b P af2db7ab1bbc490f9bceae1317438337 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: af2db7ab1bbc490f9bceae1317438337, fb643e201d404a53a320740b57e36912; no voters:
I20260806 09:29:21.142112 32000 leader_election.cc:288] T 30e2edc028634ee8b972bb1ed0fcab1b P f151cc51113947f6a0ea2f463ed20681 [CANDIDATE]: Term 1 election: Requested vote from peers af2db7ab1bbc490f9bceae1317438337 (127.20.113.2:38959), fb643e201d404a53a320740b57e36912 (127.20.113.3:35129)
I20260806 09:29:21.142752 31755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30e2edc028634ee8b972bb1ed0fcab1b" candidate_uuid: "f151cc51113947f6a0ea2f463ed20681" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb643e201d404a53a320740b57e36912"
I20260806 09:29:21.142756 31996 raft_consensus.cc:2781] T 30e2edc028634ee8b972bb1ed0fcab1b P af2db7ab1bbc490f9bceae1317438337 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 09:29:21.142815 31528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30e2edc028634ee8b972bb1ed0fcab1b" candidate_uuid: "f151cc51113947f6a0ea2f463ed20681" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af2db7ab1bbc490f9bceae1317438337"
I20260806 09:29:21.142980 31755 raft_consensus.cc:3037] T 30e2edc028634ee8b972bb1ed0fcab1b P fb643e201d404a53a320740b57e36912 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:29:21.142987 31996 raft_consensus.cc:482] T 30e2edc028634ee8b972bb1ed0fcab1b P af2db7ab1bbc490f9bceae1317438337 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 09:29:21.143110 31996 raft_consensus.cc:3037] T 30e2edc028634ee8b972bb1ed0fcab1b P af2db7ab1bbc490f9bceae1317438337 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:29:21.145339 31755 raft_consensus.cc:2454] T 30e2edc028634ee8b972bb1ed0fcab1b P fb643e201d404a53a320740b57e36912 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f151cc51113947f6a0ea2f463ed20681 in term 1.
I20260806 09:29:21.145741 31193 leader_election.cc:302] T 30e2edc028634ee8b972bb1ed0fcab1b P f151cc51113947f6a0ea2f463ed20681 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f151cc51113947f6a0ea2f463ed20681, fb643e201d404a53a320740b57e36912; no voters:
I20260806 09:29:21.146013 32000 raft_consensus.cc:2781] T 30e2edc028634ee8b972bb1ed0fcab1b P f151cc51113947f6a0ea2f463ed20681 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:29:21.146548 32000 raft_consensus.cc:686] T 30e2edc028634ee8b972bb1ed0fcab1b P f151cc51113947f6a0ea2f463ed20681 [term 1 LEADER]: Becoming Leader. State: Replica: f151cc51113947f6a0ea2f463ed20681, State: Running, Role: LEADER
I20260806 09:29:21.146533 31996 raft_consensus.cc:504] T 30e2edc028634ee8b972bb1ed0fcab1b P af2db7ab1bbc490f9bceae1317438337 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af2db7ab1bbc490f9bceae1317438337" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 38959 } } peers { permanent_uuid: "fb643e201d404a53a320740b57e36912" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 35129 } } peers { permanent_uuid: "f151cc51113947f6a0ea2f463ed20681" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 37689 } }
I20260806 09:29:21.146997 31528 raft_consensus.cc:2379] T 30e2edc028634ee8b972bb1ed0fcab1b P af2db7ab1bbc490f9bceae1317438337 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate f151cc51113947f6a0ea2f463ed20681 in current term 1: Already voted for candidate af2db7ab1bbc490f9bceae1317438337 in this term.
I20260806 09:29:21.146977 32000 consensus_queue.cc:237] T 30e2edc028634ee8b972bb1ed0fcab1b P f151cc51113947f6a0ea2f463ed20681 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af2db7ab1bbc490f9bceae1317438337" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 38959 } } peers { permanent_uuid: "fb643e201d404a53a320740b57e36912" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 35129 } } peers { permanent_uuid: "f151cc51113947f6a0ea2f463ed20681" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 37689 } }
I20260806 09:29:21.149005 31967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:21.147457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:45173 (local address 127.20.113.1:54409)
0806 09:29:21.148768 (+ 1311us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:21.148862 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:45173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1216}
I20260806 09:29:21.155968 31288 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30e2edc028634ee8b972bb1ed0fcab1b" candidate_uuid: "af2db7ab1bbc490f9bceae1317438337" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f151cc51113947f6a0ea2f463ed20681"
I20260806 09:29:21.156620 31996 leader_election.cc:288] T 30e2edc028634ee8b972bb1ed0fcab1b P af2db7ab1bbc490f9bceae1317438337 [CANDIDATE]: Term 1 election: Requested vote from peers fb643e201d404a53a320740b57e36912 (127.20.113.3:35129), f151cc51113947f6a0ea2f463ed20681 (127.20.113.1:37689)
I20260806 09:29:21.157097 31755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30e2edc028634ee8b972bb1ed0fcab1b" candidate_uuid: "af2db7ab1bbc490f9bceae1317438337" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb643e201d404a53a320740b57e36912"
I20260806 09:29:21.157397 31755 raft_consensus.cc:2379] T 30e2edc028634ee8b972bb1ed0fcab1b P fb643e201d404a53a320740b57e36912 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate af2db7ab1bbc490f9bceae1317438337 in current term 1: Already voted for candidate f151cc51113947f6a0ea2f463ed20681 in this term.
I20260806 09:29:21.157938 31429 leader_election.cc:302] T 30e2edc028634ee8b972bb1ed0fcab1b P af2db7ab1bbc490f9bceae1317438337 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: af2db7ab1bbc490f9bceae1317438337; no voters: f151cc51113947f6a0ea2f463ed20681, fb643e201d404a53a320740b57e36912
I20260806 09:29:21.158390 31996 raft_consensus.cc:2726] T 30e2edc028634ee8b972bb1ed0fcab1b P af2db7ab1bbc490f9bceae1317438337 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260806 09:29:21.158905 30815 catalog_manager.cc:5495] T 30e2edc028634ee8b972bb1ed0fcab1b P f151cc51113947f6a0ea2f463ed20681 reported cstate change: term changed from 0 to 1, leader changed from <none> to f151cc51113947f6a0ea2f463ed20681 (127.20.113.1). New cstate: current_term: 1 leader_uuid: "f151cc51113947f6a0ea2f463ed20681" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af2db7ab1bbc490f9bceae1317438337" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 38959 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fb643e201d404a53a320740b57e36912" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 35129 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f151cc51113947f6a0ea2f463ed20681" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 37689 } health_report { overall_health: HEALTHY } } }
I20260806 09:29:21.190343 31989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:21.184079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36258 (local address 127.20.113.1:37689)
0806 09:29:21.184250 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:21.184255 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:21.184273 (+ 18us) server_negotiation.cc:407] Connection header received
0806 09:29:21.184558 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:21.184564 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:21.184680 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:21.184843 (+ 163us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 09:29:21.185197 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:21.186341 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:21.187895 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:21.188597 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:21.188722 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:21.189085 (+ 363us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 09:29:21.189775 (+ 690us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 09:29:21.189912 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:21.189973 (+ 61us) server_negotiation.cc:299] Negotiation successful
0806 09:29:21.190063 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 09:29:21.194599 31755 raft_consensus.cc:1261] T 30e2edc028634ee8b972bb1ed0fcab1b P fb643e201d404a53a320740b57e36912 [term 1 FOLLOWER]: Refusing update from remote peer f151cc51113947f6a0ea2f463ed20681: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 09:29:21.195012 31528 raft_consensus.cc:1261] T 30e2edc028634ee8b972bb1ed0fcab1b P af2db7ab1bbc490f9bceae1317438337 [term 1 FOLLOWER]: Refusing update from remote peer f151cc51113947f6a0ea2f463ed20681: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 09:29:21.195304 32003 consensus_queue.cc:1035] T 30e2edc028634ee8b972bb1ed0fcab1b P f151cc51113947f6a0ea2f463ed20681 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb643e201d404a53a320740b57e36912" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 35129 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 09:29:21.195729 32000 consensus_queue.cc:1035] T 30e2edc028634ee8b972bb1ed0fcab1b P f151cc51113947f6a0ea2f463ed20681 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af2db7ab1bbc490f9bceae1317438337" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 38959 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 09:29:21.198230 31966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:21.197637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:45173 (local address 127.20.113.3:59013)
0806 09:29:21.197915 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:21.198051 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:45173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142}
I20260806 09:29:21.199944 31965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:21.199378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:45173 (local address 127.20.113.2:55513)
0806 09:29:21.199600 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:21.199721 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:45173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260806 09:29:21.204412 31966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:21.203838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:45173 (local address 127.20.113.3:58275)
0806 09:29:21.204093 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:21.204176 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:45173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147}
I20260806 09:29:21.206760 31967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:21.206246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:45173 (local address 127.20.113.1:42975)
0806 09:29:21.206503 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:21.206597 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:45173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111}
I20260806 09:29:21.210166 32008 mvcc.cc:205] Tried to move back new op lower bound from 7315491066645897216 to 7315491066463547392. Current Snapshot: MvccSnapshot[applied={T|T < 7315491066645897216}]
I20260806 09:29:21.215736 30815 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43086:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 09:29:21.216017 30815 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43086:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 09:29:21.219043 31967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:21.218294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:45173 (local address 127.20.113.1:46557)
0806 09:29:21.218776 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:21.218873 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:45173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":351}
I20260806 09:29:21.221055 32009 mvcc.cc:205] Tried to move back new op lower bound from 7315491066645897216 to 7315491066463547392. Current Snapshot: MvccSnapshot[applied={T|T < 7315491066645897216}]
I20260806 09:29:21.222970 31965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:21.221914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:45173 (local address 127.20.113.2:58873)
0806 09:29:21.222682 (+ 768us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:21.222792 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:45173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":646}
I20260806 09:29:21.226734 30815 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 377f07a0afaa4800aa98d87714775150: Sending DeleteTablet for 3 replicas of tablet 30e2edc028634ee8b972bb1ed0fcab1b
I20260806 09:29:21.228223 31502 tablet_service.cc:1514] Processing DeleteTablet for tablet 30e2edc028634ee8b972bb1ed0fcab1b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:29:21 UTC) from {username='oryx', principal='oryx/127.20.113.61@KRBTEST.COM'} at 127.0.0.1:44852
I20260806 09:29:21.228245 31713 tablet_service.cc:1514] Processing DeleteTablet for tablet 30e2edc028634ee8b972bb1ed0fcab1b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:29:21 UTC) from {username='oryx', principal='oryx/127.20.113.61@KRBTEST.COM'} at 127.0.0.1:37908
I20260806 09:29:21.228724 31261 tablet_service.cc:1514] Processing DeleteTablet for tablet 30e2edc028634ee8b972bb1ed0fcab1b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:29:21 UTC) from {username='oryx', principal='oryx/127.20.113.61@KRBTEST.COM'} at 127.0.0.1:36254
I20260806 09:29:21.229121 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 31154
I20260806 09:29:21.229210 31714 tablet_service.cc:1514] Processing DeleteTablet for tablet 30e2edc028634ee8b972bb1ed0fcab1b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:29:21 UTC) from {username='oryx', principal='oryx/127.20.113.61@KRBTEST.COM'} at 127.0.0.1:37908
I20260806 09:29:21.239611 32019 tablet_replica.cc:330] T 30e2edc028634ee8b972bb1ed0fcab1b P fb643e201d404a53a320740b57e36912: stopping tablet replica
I20260806 09:29:21.239854 32019 raft_consensus.cc:2229] T 30e2edc028634ee8b972bb1ed0fcab1b P fb643e201d404a53a320740b57e36912 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 09:29:21.240228 32019 raft_consensus.cc:2258] T 30e2edc028634ee8b972bb1ed0fcab1b P fb643e201d404a53a320740b57e36912 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 09:29:21.241698 32019 ts_tablet_manager.cc:1905] T 30e2edc028634ee8b972bb1ed0fcab1b P fb643e201d404a53a320740b57e36912: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 09:29:21.242528 32020 tablet_replica.cc:330] T 30e2edc028634ee8b972bb1ed0fcab1b P f151cc51113947f6a0ea2f463ed20681: stopping tablet replica
I20260806 09:29:21.242799 32020 raft_consensus.cc:2229] T 30e2edc028634ee8b972bb1ed0fcab1b P f151cc51113947f6a0ea2f463ed20681 [term 1 LEADER]: Raft consensus shutting down.
I20260806 09:29:21.243464 32020 raft_consensus.cc:2258] T 30e2edc028634ee8b972bb1ed0fcab1b P f151cc51113947f6a0ea2f463ed20681 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 09:29:21.245502 32018 tablet_replica.cc:330] T 30e2edc028634ee8b972bb1ed0fcab1b P af2db7ab1bbc490f9bceae1317438337: stopping tablet replica
I20260806 09:29:21.245685 32018 raft_consensus.cc:2229] T 30e2edc028634ee8b972bb1ed0fcab1b P af2db7ab1bbc490f9bceae1317438337 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 09:29:21.245977 32018 raft_consensus.cc:2258] T 30e2edc028634ee8b972bb1ed0fcab1b P af2db7ab1bbc490f9bceae1317438337 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 09:29:21.249051 32019 ts_tablet_manager.cc:1918] T 30e2edc028634ee8b972bb1ed0fcab1b P fb643e201d404a53a320740b57e36912: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 09:29:21.249073 32018 ts_tablet_manager.cc:1905] T 30e2edc028634ee8b972bb1ed0fcab1b P af2db7ab1bbc490f9bceae1317438337: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 09:29:21.249181 32019 log.cc:1196] T 30e2edc028634ee8b972bb1ed0fcab1b P fb643e201d404a53a320740b57e36912: Deleting WAL directory at /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/ts-2/wal/wals/30e2edc028634ee8b972bb1ed0fcab1b
I20260806 09:29:21.249737 32019 ts_tablet_manager.cc:1939] T 30e2edc028634ee8b972bb1ed0fcab1b P fb643e201d404a53a320740b57e36912: Deleting consensus metadata
I20260806 09:29:21.250383 32020 ts_tablet_manager.cc:1905] T 30e2edc028634ee8b972bb1ed0fcab1b P f151cc51113947f6a0ea2f463ed20681: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 09:29:21.250916 30788 catalog_manager.cc:4841] TS fb643e201d404a53a320740b57e36912 (127.20.113.3:35129): tablet 30e2edc028634ee8b972bb1ed0fcab1b (table test-table [id=88d14ccbd13040f6bf14d0e3ec362b4e]) successfully deleted
W20260806 09:29:21.252045 30788 catalog_manager.cc:4808] TS fb643e201d404a53a320740b57e36912 (127.20.113.3:35129): delete failed for tablet 30e2edc028634ee8b972bb1ed0fcab1b because the tablet was not found. No further retry: Not found: Tablet not found: 30e2edc028634ee8b972bb1ed0fcab1b
I20260806 09:29:21.257472 32018 ts_tablet_manager.cc:1918] T 30e2edc028634ee8b972bb1ed0fcab1b P af2db7ab1bbc490f9bceae1317438337: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 09:29:21.257494 32020 ts_tablet_manager.cc:1918] T 30e2edc028634ee8b972bb1ed0fcab1b P f151cc51113947f6a0ea2f463ed20681: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 09:29:21.257628 32018 log.cc:1196] T 30e2edc028634ee8b972bb1ed0fcab1b P af2db7ab1bbc490f9bceae1317438337: Deleting WAL directory at /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/ts-1/wal/wals/30e2edc028634ee8b972bb1ed0fcab1b
I20260806 09:29:21.257656 32020 log.cc:1196] T 30e2edc028634ee8b972bb1ed0fcab1b P f151cc51113947f6a0ea2f463ed20681: Deleting WAL directory at /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786008495763054-20932-0/minicluster-data/ts-0/wal/wals/30e2edc028634ee8b972bb1ed0fcab1b
I20260806 09:29:21.258147 32018 ts_tablet_manager.cc:1939] T 30e2edc028634ee8b972bb1ed0fcab1b P af2db7ab1bbc490f9bceae1317438337: Deleting consensus metadata
I20260806 09:29:21.258239 32020 ts_tablet_manager.cc:1939] T 30e2edc028634ee8b972bb1ed0fcab1b P f151cc51113947f6a0ea2f463ed20681: Deleting consensus metadata
I20260806 09:29:21.259466 30790 catalog_manager.cc:4841] TS f151cc51113947f6a0ea2f463ed20681 (127.20.113.1:37689): tablet 30e2edc028634ee8b972bb1ed0fcab1b (table test-table [id=88d14ccbd13040f6bf14d0e3ec362b4e]) successfully deleted
I20260806 09:29:21.259748 30792 catalog_manager.cc:4841] TS af2db7ab1bbc490f9bceae1317438337 (127.20.113.2:38959): tablet 30e2edc028634ee8b972bb1ed0fcab1b (table test-table [id=88d14ccbd13040f6bf14d0e3ec362b4e]) successfully deleted
I20260806 09:29:21.266930 31989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:21.229910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36264 (local address 127.20.113.1:37689)
0806 09:29:21.230595 (+ 685us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:21.230695 (+ 100us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:21.244655 (+ 13960us) server_negotiation.cc:407] Connection header received
0806 09:29:21.244848 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:21.244854 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:21.244935 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:21.245066 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:21.247936 (+ 2870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:21.249129 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:21.252861 (+ 3732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:21.253336 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:21.253488 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:21.260617 (+ 7129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:29:21.260630 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:29:21.260645 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:29:21.260696 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:29:21.264013 (+ 3317us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:21.264774 (+ 761us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:21.264781 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:21.264784 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:21.264938 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:21.265358 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:21.265367 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:21.265369 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:21.265760 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:29:21.266097 (+ 337us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:21.266377 (+ 280us) server_negotiation.cc:299] Negotiation successful
0806 09:29:21.266613 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168}
I20260806 09:29:21.267604 31364 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:29:21.333407 31988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:21.332937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:45173 (local address 127.0.0.1:60020)
0806 09:29:21.333156 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:21.333254 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:45173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260806 09:29:21.334501 31988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:21.334155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:45173 (local address 127.0.0.1:60022)
0806 09:29:21.334291 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:21.334373 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:45173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260806 09:29:21.371156 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 31154
I20260806 09:29:21.398628 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 31391
I20260806 09:29:21.413944 31983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:21.399396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44866 (local address 127.20.113.2:38959)
0806 09:29:21.399601 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:21.399606 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:21.399626 (+ 20us) server_negotiation.cc:407] Connection header received
0806 09:29:21.399793 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:21.399797 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:21.399862 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:21.399968 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:21.401472 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:21.402617 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:21.403904 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:21.404533 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:21.404647 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:21.407196 (+ 2549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:29:21.407212 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:29:21.407233 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:29:21.407271 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:29:21.411061 (+ 3790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:21.411720 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:21.411729 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:21.411732 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:21.411898 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:21.412223 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:21.412227 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:21.412231 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:21.412764 (+ 533us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:29:21.413086 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:21.413235 (+ 149us) server_negotiation.cc:299] Negotiation successful
0806 09:29:21.413463 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260806 09:29:21.414722 31602 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:29:21.534560 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 31391
I20260806 09:29:21.568230 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 31627
I20260806 09:29:21.581908 31985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:21.569103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37924 (local address 127.20.113.3:35129)
0806 09:29:21.569291 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:21.569299 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:21.569323 (+ 24us) server_negotiation.cc:407] Connection header received
0806 09:29:21.569570 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:21.569574 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:21.569637 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:21.569737 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:21.570997 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:21.571746 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:21.573065 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:21.573461 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:21.573593 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:21.576140 (+ 2547us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:29:21.576157 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:29:21.576173 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:29:21.576207 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:29:21.579008 (+ 2801us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:21.579692 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:21.579700 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:21.579702 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:21.579829 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:21.580276 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:21.580281 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:21.580286 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:21.580708 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:29:21.581034 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:21.581326 (+ 292us) server_negotiation.cc:299] Negotiation successful
0806 09:29:21.581569 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 09:29:21.582633 31814 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:29:21.675611 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 31627
I20260806 09:29:21.704648 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 30601
I20260806 09:29:21.705823 30861 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:29:21.831633 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 30601
W20260806 09:29:21.863828 31075 connection.cc:535] server connection from 127.0.0.1:60318 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 09:29:21.864168 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 30761
I20260806 09:29:21.865290 31144 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:29:21.983151 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 30761
2026-08-06T09:29:22Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (12259 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786008495763054-20932-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 09:29:22 dist-test-slave-45dx krb5kdc[32029](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 09:29:22 dist-test-slave-45dx krb5kdc[32029](info): set up 2 sockets
Aug 06 09:29:22 dist-test-slave-45dx krb5kdc[32029](info): commencing operation
krb5kdc: starting...
W20260806 09:29:23.296545 20932 mini_kdc.cc:121] Time spent starting KDC: real 1.255s user 0.005s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786008495763054-20932-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786008495763054-20932-0/krb5kdc/test-user.keytab.
Aug 06 09:29:23 dist-test-slave-45dx krb5kdc[32029](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008563, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T09:29:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T09:29:23Z Disabled control of system clock
WARNING: no policy specified for oryx/127.20.113.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.113.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.113.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786008495763054-20932-0/krb5kdc/oryx_127.20.113.62.keytab.
Entry for principal oryx/127.20.113.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786008495763054-20932-0/krb5kdc/oryx_127.20.113.62.keytab.
I20260806 09:29:23.569288 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786008495763054-20932-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786008495763054-20932-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786008495763054-20932-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.113.62:45237
--webserver_interface=127.20.113.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786008495763054-20932-0/krb5kdc/oryx_127.20.113.62.keytab
--principal=oryx/127.20.113.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:37413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.113.62:45237
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:29:23.898620 32056 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:29:23.898911 32056 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:29:23.899003 32056 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:29:23.906003 32056 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 09:29:23.906095 32056 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 09:29:23.906153 32056 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:29:23.906206 32056 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:29:23.906250 32056 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 09:29:23.906307 32056 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 09:29:23.915714 32056 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:37413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786008495763054-20932-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786008495763054-20932-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.113.62:45237
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786008495763054-20932-0/krb5kdc/oryx_127.20.113.62.keytab
--principal=oryx/127.20.113.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.113.62:45237
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.113.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786008495763054-20932-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:29:23.916924 32056 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:29:23.918119 32056 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:29:23.929704 32067 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:29:23.929764 32065 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:29:23.929785 32064 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:29:23.930500 32056 server_base.cc:1034] running on GCE node
I20260806 09:29:23.931320 32056 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:29:23.932637 32056 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:29:23.933849 32056 hybrid_clock.cc:648] HybridClock initialized: now 1786008563933788 us; error 32 us; skew 500 ppm
I20260806 09:29:23.934195 32056 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:29:23 dist-test-slave-45dx krb5kdc[32029](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008563, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.113.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:29:23.937372 32056 init.cc:377] Logged in from keytab as oryx/127.20.113.62@KRBTEST.COM (short username oryx)
I20260806 09:29:23.938944 32056 webserver.cc:460] Webserver started at http://127.20.113.62:33347/ using document root <none> and password file <none>
I20260806 09:29:23.939692 32056 fs_manager.cc:359] Metadata directory not provided
I20260806 09:29:23.939826 32056 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:29:23.940176 32056 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:29:23.943256 32056 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786008495763054-20932-0/minicluster-data/master-0/data/instance:
uuid: "81679e1de52c40d883ea9f9173070aa7"
format_stamp: "Formatted at 2026-08-06 09:29:23 on dist-test-slave-45dx"
server_key: "78ffb748f1f654adc27c0129c555ddb4"
server_key_iv: "c5f9f49e07c727a15d5825ab9c67b6bc"
server_key_version: "encryptionkey@0"
I20260806 09:29:23.944105 32056 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786008495763054-20932-0/minicluster-data/master-0/wal/instance:
uuid: "81679e1de52c40d883ea9f9173070aa7"
format_stamp: "Formatted at 2026-08-06 09:29:23 on dist-test-slave-45dx"
server_key: "78ffb748f1f654adc27c0129c555ddb4"
server_key_iv: "c5f9f49e07c727a15d5825ab9c67b6bc"
server_key_version: "encryptionkey@0"
I20260806 09:29:23.949573 32056 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260806 09:29:23.953109 32074 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:29:23.954443 32056 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 09:29:23.954633 32056 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786008495763054-20932-0/minicluster-data/master-0/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786008495763054-20932-0/minicluster-data/master-0/wal
uuid: "81679e1de52c40d883ea9f9173070aa7"
format_stamp: "Formatted at 2026-08-06 09:29:23 on dist-test-slave-45dx"
server_key: "78ffb748f1f654adc27c0129c555ddb4"
server_key_iv: "c5f9f49e07c727a15d5825ab9c67b6bc"
server_key_version: "encryptionkey@0"
I20260806 09:29:23.954854 32056 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786008495763054-20932-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786008495763054-20932-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786008495763054-20932-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:29:23.989936 32056 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:29:23.994001 32056 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:29:23.994287 32056 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:29:24.016621 32056 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.62:45237
I20260806 09:29:24.016640 32168 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.62:45237 every 8 connection(s)
I20260806 09:29:24.017961 32056 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
I20260806 09:29:24.021575 32171 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:29:24.025753 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 32056
I20260806 09:29:24.025967 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786008495763054-20932-0/minicluster-data/master-0/wal/instance
I20260806 09:29:24.026599 20932 external_mini_cluster.cc:1428] Setting key 52d59d62dbdc7e87e8562b03ef7ff79e
I20260806 09:29:24.033668 32171 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81679e1de52c40d883ea9f9173070aa7: Bootstrap starting.
I20260806 09:29:24.037369 32171 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 81679e1de52c40d883ea9f9173070aa7: Neither blocks nor log segments found. Creating new log.
I20260806 09:29:24.038465 32171 log.cc:824] T 00000000000000000000000000000000 P 81679e1de52c40d883ea9f9173070aa7: Log is configured to *not* fsync() on all Append() calls
I20260806 09:29:24.041188 32171 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81679e1de52c40d883ea9f9173070aa7: No bootstrap required, opened a new log
Aug 06 09:29:24 dist-test-slave-45dx krb5kdc[32029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008563, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM
I20260806 09:29:24.049244 32171 raft_consensus.cc:348] T 00000000000000000000000000000000 P 81679e1de52c40d883ea9f9173070aa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81679e1de52c40d883ea9f9173070aa7" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 45237 } }
I20260806 09:29:24.049628 32171 raft_consensus.cc:374] T 00000000000000000000000000000000 P 81679e1de52c40d883ea9f9173070aa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:29:24.049760 32171 raft_consensus.cc:729] T 00000000000000000000000000000000 P 81679e1de52c40d883ea9f9173070aa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81679e1de52c40d883ea9f9173070aa7, State: Initialized, Role: FOLLOWER
I20260806 09:29:24.049790 32174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:24.028505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53428 (local address 127.20.113.62:45237)
0806 09:29:24.029379 (+ 874us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:24.029394 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:24.029441 (+ 47us) server_negotiation.cc:407] Connection header received
0806 09:29:24.031957 (+ 2516us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:24.031985 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:24.032499 (+ 514us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:24.032881 (+ 382us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:24.034219 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:24.036234 (+ 2015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:24.037578 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:24.038302 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:24.038503 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:24.042518 (+ 4015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:29:24.042525 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:29:24.042538 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:29:24.042572 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:29:24.045823 (+ 3251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:24.046487 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:24.046493 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:24.046499 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:24.046663 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:24.047069 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:24.047073 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:24.047075 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:24.047613 (+ 538us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:29:24.047939 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:24.048405 (+ 466us) server_negotiation.cc:299] Negotiation successful
0806 09:29:24.048787 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":628,"thread_start_us":380,"threads_started":1}
I20260806 09:29:24.050463 32171 consensus_queue.cc:260] T 00000000000000000000000000000000 P 81679e1de52c40d883ea9f9173070aa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81679e1de52c40d883ea9f9173070aa7" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 45237 } }
I20260806 09:29:24.050760 32171 raft_consensus.cc:388] T 00000000000000000000000000000000 P 81679e1de52c40d883ea9f9173070aa7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 09:29:24.050906 32171 raft_consensus.cc:482] T 00000000000000000000000000000000 P 81679e1de52c40d883ea9f9173070aa7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 09:29:24.051103 32171 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 81679e1de52c40d883ea9f9173070aa7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:29:24.053627 32171 raft_consensus.cc:504] T 00000000000000000000000000000000 P 81679e1de52c40d883ea9f9173070aa7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81679e1de52c40d883ea9f9173070aa7" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 45237 } }
I20260806 09:29:24.054152 32171 leader_election.cc:302] T 00000000000000000000000000000000 P 81679e1de52c40d883ea9f9173070aa7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 81679e1de52c40d883ea9f9173070aa7; no voters:
I20260806 09:29:24.054867 32171 leader_election.cc:288] T 00000000000000000000000000000000 P 81679e1de52c40d883ea9f9173070aa7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 09:29:24.054970 32179 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 81679e1de52c40d883ea9f9173070aa7 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:29:24.055704 32179 raft_consensus.cc:686] T 00000000000000000000000000000000 P 81679e1de52c40d883ea9f9173070aa7 [term 1 LEADER]: Becoming Leader. State: Replica: 81679e1de52c40d883ea9f9173070aa7, State: Running, Role: LEADER
I20260806 09:29:24.056340 32179 consensus_queue.cc:237] T 00000000000000000000000000000000 P 81679e1de52c40d883ea9f9173070aa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81679e1de52c40d883ea9f9173070aa7" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 45237 } }
I20260806 09:29:24.057143 32171 sys_catalog.cc:564] T 00000000000000000000000000000000 P 81679e1de52c40d883ea9f9173070aa7 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 09:29:24.059288 32181 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81679e1de52c40d883ea9f9173070aa7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 81679e1de52c40d883ea9f9173070aa7. Latest consensus state: current_term: 1 leader_uuid: "81679e1de52c40d883ea9f9173070aa7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81679e1de52c40d883ea9f9173070aa7" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 45237 } } }
I20260806 09:29:24.059700 32181 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81679e1de52c40d883ea9f9173070aa7 [sys.catalog]: This master's current role is: LEADER
I20260806 09:29:24.060187 32180 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81679e1de52c40d883ea9f9173070aa7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "81679e1de52c40d883ea9f9173070aa7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81679e1de52c40d883ea9f9173070aa7" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 45237 } } }
I20260806 09:29:24.060524 32180 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81679e1de52c40d883ea9f9173070aa7 [sys.catalog]: This master's current role is: LEADER
I20260806 09:29:24.062950 32186 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 09:29:24.068351 32186 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 09:29:24.078156 32186 catalog_manager.cc:1347] Generated new cluster ID: ebfe4ae7a29b4a83a9041170365ea926
I20260806 09:29:24.078284 32186 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 09:29:24.116967 32186 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 09:29:24.118882 32186 catalog_manager.cc:1504] Loading token signing keys...
I20260806 09:29:24.131808 32186 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 81679e1de52c40d883ea9f9173070aa7: Generated new TSK 0
I20260806 09:29:24.132861 32186 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.20.113.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.113.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.113.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786008495763054-20932-0/krb5kdc/oryx_127.20.113.1.keytab.
Entry for principal oryx/127.20.113.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786008495763054-20932-0/krb5kdc/oryx_127.20.113.1.keytab.
I20260806 09:29:24.212496 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786008495763054-20932-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786008495763054-20932-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786008495763054-20932-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786008495763054-20932-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.113.1:0
--local_ip_for_outbound_sockets=127.20.113.1
--webserver_interface=127.20.113.1
--webserver_port=0
--tserver_master_addrs=127.20.113.62:45237
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786008495763054-20932-0/krb5kdc/oryx_127.20.113.1.keytab
--principal=oryx/127.20.113.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:37413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:29:24.563906 32209 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:29:24.564227 32209 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:29:24.564428 32209 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:29:24.571485 32209 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:29:24.571646 32209 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:29:24.571795 32209 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 09:29:24.571936 32209 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.113.1
I20260806 09:29:24.581405 32209 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:37413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786008495763054-20932-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786008495763054-20932-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786008495763054-20932-0/krb5kdc/oryx_127.20.113.1.keytab
--principal=oryx/127.20.113.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.113.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786008495763054-20932-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.113.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.20.113.62:45237
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.113.1
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786008495763054-20932-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:29:24.582729 32209 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:29:24.584101 32209 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:29:24.597411 32216 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:29:24.597385 32220 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:29:24.597318 32218 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:29:24.598239 32209 server_base.cc:1034] running on GCE node
I20260806 09:29:24.598878 32209 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:29:24.599735 32209 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:29:24.600991 32209 hybrid_clock.cc:648] HybridClock initialized: now 1786008564600931 us; error 38 us; skew 500 ppm
I20260806 09:29:24.601317 32209 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:29:24 dist-test-slave-45dx krb5kdc[32029](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008564, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.113.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:29:24.604524 32209 init.cc:377] Logged in from keytab as oryx/127.20.113.1@KRBTEST.COM (short username oryx)
I20260806 09:29:24.605924 32209 webserver.cc:460] Webserver started at http://127.20.113.1:38759/ using document root <none> and password file <none>
I20260806 09:29:24.606618 32209 fs_manager.cc:359] Metadata directory not provided
I20260806 09:29:24.606755 32209 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:29:24.607095 32209 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:29:24.610018 32209 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786008495763054-20932-0/minicluster-data/ts-0/data/instance:
uuid: "a27e10c55c654384ae155607041dad72"
format_stamp: "Formatted at 2026-08-06 09:29:24 on dist-test-slave-45dx"
server_key: "5581b6ad01d1f269d48eb33975cd3270"
server_key_iv: "936f3a7583fddd71cef8f9e63c13fe79"
server_key_version: "encryptionkey@0"
I20260806 09:29:24.610829 32209 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786008495763054-20932-0/minicluster-data/ts-0/wal/instance:
uuid: "a27e10c55c654384ae155607041dad72"
format_stamp: "Formatted at 2026-08-06 09:29:24 on dist-test-slave-45dx"
server_key: "5581b6ad01d1f269d48eb33975cd3270"
server_key_iv: "936f3a7583fddd71cef8f9e63c13fe79"
server_key_version: "encryptionkey@0"
I20260806 09:29:24.616184 32209 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 09:29:24.619865 32227 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:29:24.621263 32209 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 09:29:24.621451 32209 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786008495763054-20932-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786008495763054-20932-0/minicluster-data/ts-0/wal
uuid: "a27e10c55c654384ae155607041dad72"
format_stamp: "Formatted at 2026-08-06 09:29:24 on dist-test-slave-45dx"
server_key: "5581b6ad01d1f269d48eb33975cd3270"
server_key_iv: "936f3a7583fddd71cef8f9e63c13fe79"
server_key_version: "encryptionkey@0"
I20260806 09:29:24.621712 32209 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786008495763054-20932-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786008495763054-20932-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786008495763054-20932-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:29:24.641108 32209 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:29:24.644991 32209 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:29:24.645213 32209 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:29:24.647599 32209 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:29:24.647727 32209 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:29:24.647876 32209 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:29:24.647948 32209 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 09:29:24 dist-test-slave-45dx krb5kdc[32029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008564, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.113.1@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM
I20260806 09:29:24.673421 32241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:24.648391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.1:42501 (local address 127.20.113.62:45237)
0806 09:29:24.650744 (+ 2353us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:24.650752 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:24.650778 (+ 26us) server_negotiation.cc:407] Connection header received
0806 09:29:24.652723 (+ 1945us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:24.652729 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:24.652814 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:24.652956 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:24.655229 (+ 2273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:24.656375 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:24.659154 (+ 2779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:24.659855 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:24.660022 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:24.665154 (+ 5132us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:29:24.665164 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:29:24.665171 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:29:24.665223 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:29:24.669193 (+ 3970us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:24.670160 (+ 967us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:24.670167 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:24.670170 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:24.670338 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:24.671107 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:24.671114 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:24.671120 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:24.671550 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:29:24.671708 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:24.672971 (+ 1263us) server_negotiation.cc:299] Negotiation successful
0806 09:29:24.673175 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2221,"thread_start_us":2115,"threads_started":1}
I20260806 09:29:24.675393 32242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:24.649006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:45237 (local address 127.20.113.1:42501)
0806 09:29:24.650092 (+ 1086us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:24.650166 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:24.652429 (+ 2263us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:24.653206 (+ 777us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:24.653214 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:24.653976 (+ 762us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:29:24.655010 (+ 1034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:24.655043 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:24.656580 (+ 1537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:24.656595 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:24.658984 (+ 2389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:24.659003 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:24.660226 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:24.660238 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:24.660518 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:24.661217 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:29:24.661244 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:29:24.664794 (+ 3550us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:29:24.669354 (+ 4560us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:29:24.669366 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:29:24.669385 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:29:24.670020 (+ 635us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:29:24.670502 (+ 482us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:29:24.670514 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:29:24.670520 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:29:24.670907 (+ 387us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:29:24.671805 (+ 898us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:29:24.671820 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:29:24.672455 (+ 635us) client_negotiation.cc:769] Sending connection context
0806 09:29:24.672841 (+ 386us) client_negotiation.cc:240] Negotiation successful
0806 09:29:24.673506 (+ 665us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":726,"thread_start_us":447,"threads_started":1}
I20260806 09:29:24.695766 32241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:24.681623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.1:50227 (local address 127.20.113.62:45237)
0806 09:29:24.681831 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:24.681839 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:24.682680 (+ 841us) server_negotiation.cc:407] Connection header received
0806 09:29:24.682924 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:24.682929 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:24.683034 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:24.683132 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:24.684018 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:24.684919 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:24.687024 (+ 2105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:24.687488 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:24.687615 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:24.689330 (+ 1715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:29:24.689338 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:29:24.689343 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:29:24.689387 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:29:24.692599 (+ 3212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:24.693429 (+ 830us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:24.693435 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:24.693438 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:24.693602 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:24.694022 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:24.694028 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:24.694032 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:24.694449 (+ 417us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:29:24.694649 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:24.695162 (+ 513us) server_negotiation.cc:299] Negotiation successful
0806 09:29:24.695388 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 09:29:24.695943 32360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:24.681685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:45237 (local address 127.20.113.1:50227)
0806 09:29:24.682535 (+ 850us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:24.682596 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:24.682792 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:24.683157 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:24.683162 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:24.683597 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:29:24.683876 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:24.683885 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:24.685110 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:24.685114 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:24.686871 (+ 1757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:24.686887 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:24.687679 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:24.687688 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:24.687847 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:24.688472 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:29:24.688487 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:29:24.689141 (+ 654us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:29:24.692747 (+ 3606us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:29:24.692752 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:29:24.692755 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:29:24.693234 (+ 479us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:29:24.693703 (+ 469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:29:24.693709 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:29:24.693712 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:29:24.693871 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:29:24.694585 (+ 714us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:29:24.694593 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:29:24.694845 (+ 252us) client_negotiation.cc:769] Sending connection context
0806 09:29:24.695044 (+ 199us) client_negotiation.cc:240] Negotiation successful
0806 09:29:24.695412 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":673,"thread_start_us":521,"threads_started":1}
I20260806 09:29:24.697711 32209 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.1:33839
I20260806 09:29:24.697732 32408 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.1:33839 every 8 connection(s)
I20260806 09:29:24.699242 32209 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786008495763054-20932-0/minicluster-data/ts-0/data/info.pb
I20260806 09:29:24.701398 32410 heartbeater.cc:347] Connected to a master server at 127.20.113.62:45237
I20260806 09:29:24.701635 32410 heartbeater.cc:464] Registering TS with master...
I20260806 09:29:24.702317 32410 heartbeater.cc:511] Master 127.20.113.62:45237 requested a full tablet report, sending...
I20260806 09:29:24.703357 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 32209
I20260806 09:29:24.703567 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786008495763054-20932-0/minicluster-data/ts-0/wal/instance
I20260806 09:29:24.704242 20932 external_mini_cluster.cc:1428] Setting key 7fab9c872bfbd843fea499135fe7185a
I20260806 09:29:24.704465 32097 ts_manager.cc:194] Registered new tserver with Master: a27e10c55c654384ae155607041dad72 (127.20.113.1:33839)
I20260806 09:29:24.706082 32097 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.20.113.1@KRBTEST.COM'} at 127.20.113.1:42501
WARNING: no policy specified for oryx/127.20.113.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.113.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.113.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786008495763054-20932-0/krb5kdc/oryx_127.20.113.2.keytab.
Entry for principal oryx/127.20.113.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786008495763054-20932-0/krb5kdc/oryx_127.20.113.2.keytab.
I20260806 09:29:24.773222 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786008495763054-20932-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786008495763054-20932-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786008495763054-20932-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786008495763054-20932-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.113.2:0
--local_ip_for_outbound_sockets=127.20.113.2
--webserver_interface=127.20.113.2
--webserver_port=0
--tserver_master_addrs=127.20.113.62:45237
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786008495763054-20932-0/krb5kdc/oryx_127.20.113.2.keytab
--principal=oryx/127.20.113.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:37413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:29:25.114133 32418 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:29:25.114481 32418 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:29:25.114662 32418 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:29:25.122040 32418 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:29:25.122185 32418 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:29:25.122325 32418 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 09:29:25.122457 32418 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.113.2
I20260806 09:29:25.132207 32418 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:37413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786008495763054-20932-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786008495763054-20932-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786008495763054-20932-0/krb5kdc/oryx_127.20.113.2.keytab
--principal=oryx/127.20.113.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.113.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786008495763054-20932-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.113.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.20.113.62:45237
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.113.2
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786008495763054-20932-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:29:25.133499 32418 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:29:25.134876 32418 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:29:25.148372 32427 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:29:25.148430 32431 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:29:25.148452 32428 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:29:25.149354 32418 server_base.cc:1034] running on GCE node
I20260806 09:29:25.150012 32418 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:29:25.150882 32418 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:29:25.152127 32418 hybrid_clock.cc:648] HybridClock initialized: now 1786008565152075 us; error 43 us; skew 500 ppm
I20260806 09:29:25.152472 32418 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:29:25 dist-test-slave-45dx krb5kdc[32029](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008565, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.113.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:29:25.156117 32418 init.cc:377] Logged in from keytab as oryx/127.20.113.2@KRBTEST.COM (short username oryx)
I20260806 09:29:25.157711 32418 webserver.cc:460] Webserver started at http://127.20.113.2:37107/ using document root <none> and password file <none>
I20260806 09:29:25.158612 32418 fs_manager.cc:359] Metadata directory not provided
I20260806 09:29:25.158738 32418 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:29:25.159101 32418 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:29:25.162180 32418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786008495763054-20932-0/minicluster-data/ts-1/data/instance:
uuid: "d9485c4273194777b6ccf02c3e050e6f"
format_stamp: "Formatted at 2026-08-06 09:29:25 on dist-test-slave-45dx"
server_key: "19253e30ad0b6db41aff06ac28788d7a"
server_key_iv: "2f2bc26f0cfb866b475964858b83016f"
server_key_version: "encryptionkey@0"
I20260806 09:29:25.163095 32418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786008495763054-20932-0/minicluster-data/ts-1/wal/instance:
uuid: "d9485c4273194777b6ccf02c3e050e6f"
format_stamp: "Formatted at 2026-08-06 09:29:25 on dist-test-slave-45dx"
server_key: "19253e30ad0b6db41aff06ac28788d7a"
server_key_iv: "2f2bc26f0cfb866b475964858b83016f"
server_key_version: "encryptionkey@0"
I20260806 09:29:25.168368 32418 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260806 09:29:25.171890 32445 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:29:25.173246 32418 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 09:29:25.173424 32418 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786008495763054-20932-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786008495763054-20932-0/minicluster-data/ts-1/wal
uuid: "d9485c4273194777b6ccf02c3e050e6f"
format_stamp: "Formatted at 2026-08-06 09:29:25 on dist-test-slave-45dx"
server_key: "19253e30ad0b6db41aff06ac28788d7a"
server_key_iv: "2f2bc26f0cfb866b475964858b83016f"
server_key_version: "encryptionkey@0"
I20260806 09:29:25.173640 32418 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786008495763054-20932-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786008495763054-20932-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786008495763054-20932-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:29:25.187067 32418 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:29:25.191031 32418 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:29:25.191304 32418 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:29:25.194015 32418 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:29:25.194147 32418 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:29:25.194322 32418 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:29:25.194401 32418 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 09:29:25 dist-test-slave-45dx krb5kdc[32029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008565, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.113.2@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM
I20260806 09:29:25.220448 32241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:25.194387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.2:43775 (local address 127.20.113.62:45237)
0806 09:29:25.194576 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:25.194583 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:25.196132 (+ 1549us) server_negotiation.cc:407] Connection header received
0806 09:29:25.199196 (+ 3064us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:25.199204 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:25.199307 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:25.199416 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:25.201980 (+ 2564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:25.203062 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:25.205802 (+ 2740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:25.206380 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:25.206532 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:25.211862 (+ 5330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:29:25.211866 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:29:25.211870 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:29:25.211906 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:29:25.216091 (+ 4185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:25.217198 (+ 1107us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:25.217203 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:25.217207 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:25.217408 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:25.218080 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:25.218086 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:25.218092 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:25.218439 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:29:25.218647 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:25.219799 (+ 1152us) server_negotiation.cc:299] Negotiation successful
0806 09:29:25.220044 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 09:29:25.222162 32461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:25.195026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:45237 (local address 127.20.113.2:43775)
0806 09:29:25.195946 (+ 920us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:25.195999 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:25.198839 (+ 2840us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:25.199839 (+ 1000us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:25.199852 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:25.200751 (+ 899us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:29:25.201736 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:25.201781 (+ 45us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:25.203226 (+ 1445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:25.203234 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:25.205619 (+ 2385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:25.205635 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:25.206672 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:25.206679 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:25.207085 (+ 406us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:25.207925 (+ 840us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:29:25.207962 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:29:25.211623 (+ 3661us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:29:25.216253 (+ 4630us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:29:25.216260 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:29:25.216272 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:29:25.217063 (+ 791us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:29:25.217596 (+ 533us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:29:25.217603 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:29:25.217606 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:29:25.217882 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:29:25.218749 (+ 867us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:29:25.218760 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:29:25.219259 (+ 499us) client_negotiation.cc:769] Sending connection context
0806 09:29:25.219661 (+ 402us) client_negotiation.cc:240] Negotiation successful
0806 09:29:25.220235 (+ 574us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":611,"thread_start_us":384,"threads_started":1}
I20260806 09:29:25.243045 32241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:25.227761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.2:57253 (local address 127.20.113.62:45237)
0806 09:29:25.228017 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:25.228026 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:25.228672 (+ 646us) server_negotiation.cc:407] Connection header received
0806 09:29:25.228803 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:25.228809 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:25.228914 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:25.229057 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:25.230676 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:25.232025 (+ 1349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:25.233883 (+ 1858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:25.234520 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:25.234653 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:25.236427 (+ 1774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:29:25.236433 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:29:25.236437 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:29:25.236478 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:29:25.239638 (+ 3160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:25.240599 (+ 961us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:25.240604 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:25.240608 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:25.240770 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:25.241290 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:25.241296 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:25.241302 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:25.241807 (+ 505us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:29:25.241975 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:25.242584 (+ 609us) server_negotiation.cc:299] Negotiation successful
0806 09:29:25.242816 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 09:29:25.243654 32556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:25.227803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:45237 (local address 127.20.113.2:57253)
0806 09:29:25.228557 (+ 754us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:25.228586 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:25.228697 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:25.229111 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:25.229116 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:25.229824 (+ 708us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:29:25.230475 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:25.230492 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:25.232177 (+ 1685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:25.232181 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:25.233712 (+ 1531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:25.233737 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:25.234730 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:25.234736 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:25.234881 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:25.235511 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:29:25.235528 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:29:25.236219 (+ 691us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:29:25.239773 (+ 3554us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:29:25.239778 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:29:25.239782 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:29:25.240480 (+ 698us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:29:25.240901 (+ 421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:29:25.240909 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:29:25.240916 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:29:25.241123 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:29:25.241939 (+ 816us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:29:25.241944 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:29:25.242230 (+ 286us) client_negotiation.cc:769] Sending connection context
0806 09:29:25.242449 (+ 219us) client_negotiation.cc:240] Negotiation successful
0806 09:29:25.242889 (+ 440us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":571,"thread_start_us":386,"threads_started":1}
I20260806 09:29:25.250002 32418 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.2:45517
I20260806 09:29:25.250025 32622 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.2:45517 every 8 connection(s)
I20260806 09:29:25.251181 32418 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786008495763054-20932-0/minicluster-data/ts-1/data/info.pb
I20260806 09:29:25.252035 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 32418
I20260806 09:29:25.252225 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786008495763054-20932-0/minicluster-data/ts-1/wal/instance
I20260806 09:29:25.252840 20932 external_mini_cluster.cc:1428] Setting key 330f141a8721479e30d52c860252a750
I20260806 09:29:25.254217 32627 heartbeater.cc:347] Connected to a master server at 127.20.113.62:45237
I20260806 09:29:25.254483 32627 heartbeater.cc:464] Registering TS with master...
I20260806 09:29:25.255194 32627 heartbeater.cc:511] Master 127.20.113.62:45237 requested a full tablet report, sending...
I20260806 09:29:25.256672 32095 ts_manager.cc:194] Registered new tserver with Master: d9485c4273194777b6ccf02c3e050e6f (127.20.113.2:45517)
I20260806 09:29:25.257848 32095 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.20.113.2@KRBTEST.COM'} at 127.20.113.2:43775
WARNING: no policy specified for oryx/127.20.113.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.113.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.113.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786008495763054-20932-0/krb5kdc/oryx_127.20.113.3.keytab.
Entry for principal oryx/127.20.113.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786008495763054-20932-0/krb5kdc/oryx_127.20.113.3.keytab.
I20260806 09:29:25.315934 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786008495763054-20932-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786008495763054-20932-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786008495763054-20932-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786008495763054-20932-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.113.3:0
--local_ip_for_outbound_sockets=127.20.113.3
--webserver_interface=127.20.113.3
--webserver_port=0
--tserver_master_addrs=127.20.113.62:45237
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786008495763054-20932-0/krb5kdc/oryx_127.20.113.3.keytab
--principal=oryx/127.20.113.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:37413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:29:25.659809 32631 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:29:25.660140 32631 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:29:25.660235 32631 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:29:25.667644 32631 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:29:25.667750 32631 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:29:25.667838 32631 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 09:29:25.667915 32631 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.113.3
I20260806 09:29:25.677670 32631 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:37413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786008495763054-20932-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786008495763054-20932-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786008495763054-20932-0/krb5kdc/oryx_127.20.113.3.keytab
--principal=oryx/127.20.113.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.113.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786008495763054-20932-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.113.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.20.113.62:45237
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.113.3
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786008495763054-20932-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:29:25.679059 32631 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:29:25.680428 32631 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:29:25.693817 32638 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:29:25.693851 32639 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:29:25.693907 32641 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:29:25.694729 32631 server_base.cc:1034] running on GCE node
I20260806 09:29:25.695376 32631 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:29:25.696277 32631 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:29:25.697582 32631 hybrid_clock.cc:648] HybridClock initialized: now 1786008565697524 us; error 38 us; skew 500 ppm
I20260806 09:29:25.697893 32631 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:29:25 dist-test-slave-45dx krb5kdc[32029](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008565, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.113.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:29:25.701087 32631 init.cc:377] Logged in from keytab as oryx/127.20.113.3@KRBTEST.COM (short username oryx)
I20260806 09:29:25.702646 32631 webserver.cc:460] Webserver started at http://127.20.113.3:35853/ using document root <none> and password file <none>
I20260806 09:29:25.703320 32631 fs_manager.cc:359] Metadata directory not provided
I20260806 09:29:25.703445 32631 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:29:25.703783 32631 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:29:25.706735 32631 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786008495763054-20932-0/minicluster-data/ts-2/data/instance:
uuid: "d6930c871eec4d2f9c6efa9c8e742cba"
format_stamp: "Formatted at 2026-08-06 09:29:25 on dist-test-slave-45dx"
server_key: "e04a0efe445b93cc93b6fd9472a01aec"
server_key_iv: "3bb46ff33df7ad4e3b481ddd57fe1fd0"
server_key_version: "encryptionkey@0"
I20260806 09:29:25.707589 32631 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786008495763054-20932-0/minicluster-data/ts-2/wal/instance:
uuid: "d6930c871eec4d2f9c6efa9c8e742cba"
format_stamp: "Formatted at 2026-08-06 09:29:25 on dist-test-slave-45dx"
server_key: "e04a0efe445b93cc93b6fd9472a01aec"
server_key_iv: "3bb46ff33df7ad4e3b481ddd57fe1fd0"
server_key_version: "encryptionkey@0"
I20260806 09:29:25.708838 32410 heartbeater.cc:503] Master 127.20.113.62:45237 was elected leader, sending a full tablet report...
I20260806 09:29:25.713757 32631 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260806 09:29:25.717447 32650 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:29:25.718812 32631 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 09:29:25.719012 32631 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786008495763054-20932-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786008495763054-20932-0/minicluster-data/ts-2/wal
uuid: "d6930c871eec4d2f9c6efa9c8e742cba"
format_stamp: "Formatted at 2026-08-06 09:29:25 on dist-test-slave-45dx"
server_key: "e04a0efe445b93cc93b6fd9472a01aec"
server_key_iv: "3bb46ff33df7ad4e3b481ddd57fe1fd0"
server_key_version: "encryptionkey@0"
I20260806 09:29:25.719271 32631 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786008495763054-20932-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786008495763054-20932-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786008495763054-20932-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:29:25.730396 32631 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:29:25.734264 32631 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:29:25.734549 32631 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:29:25.737344 32631 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:29:25.737476 32631 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:29:25.737668 32631 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:29:25.737751 32631 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 09:29:25 dist-test-slave-45dx krb5kdc[32029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008565, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.113.3@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM
I20260806 09:29:25.762358 32241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:25.737687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.3:53321 (local address 127.20.113.62:45237)
0806 09:29:25.737871 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:25.737877 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:25.739401 (+ 1524us) server_negotiation.cc:407] Connection header received
0806 09:29:25.741896 (+ 2495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:25.741908 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:25.742094 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:25.742266 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:25.745157 (+ 2891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:25.746261 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:25.748677 (+ 2416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:25.749273 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:25.749397 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:25.754281 (+ 4884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:29:25.754290 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:29:25.754298 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:29:25.754351 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:29:25.758211 (+ 3860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:25.759469 (+ 1258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:25.759477 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:25.759481 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:25.759632 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:25.760175 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:25.760181 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:25.760184 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:25.760555 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:29:25.760724 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:25.761921 (+ 1197us) server_negotiation.cc:299] Negotiation successful
0806 09:29:25.762096 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 09:29:25.764431 32661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:25.738287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:45237 (local address 127.20.113.3:53321)
0806 09:29:25.739196 (+ 909us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:25.739244 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:25.741418 (+ 2174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:25.742679 (+ 1261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:25.742693 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:25.743768 (+ 1075us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:29:25.744894 (+ 1126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:25.744929 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:25.746447 (+ 1518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:25.746451 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:25.748539 (+ 2088us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:25.748551 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:25.749530 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:25.749546 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:25.749921 (+ 375us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:25.750648 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:29:25.750687 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:29:25.753915 (+ 3228us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:29:25.758401 (+ 4486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:29:25.758410 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:29:25.758467 (+ 57us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:29:25.759287 (+ 820us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:29:25.759754 (+ 467us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:29:25.759758 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:29:25.759765 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:29:25.760052 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:29:25.760707 (+ 655us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:29:25.760715 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:29:25.761304 (+ 589us) client_negotiation.cc:769] Sending connection context
0806 09:29:25.761742 (+ 438us) client_negotiation.cc:240] Negotiation successful
0806 09:29:25.762475 (+ 733us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":633,"thread_start_us":394,"threads_started":1}
I20260806 09:29:25.784045 32631 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.3:38869
I20260806 09:29:25.784057 349 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.3:38869 every 8 connection(s)
I20260806 09:29:25.785645 32631 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786008495763054-20932-0/minicluster-data/ts-2/data/info.pb
I20260806 09:29:25.788103 352 heartbeater.cc:347] Connected to a master server at 127.20.113.62:45237
I20260806 09:29:25.788239 305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:25.771829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:45237 (local address 127.20.113.3:32989)
0806 09:29:25.772539 (+ 710us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:25.772563 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:25.772716 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:25.773310 (+ 594us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:25.773321 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:25.774052 (+ 731us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:29:25.774506 (+ 454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:25.774523 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:25.775694 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:25.775699 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:25.777033 (+ 1334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:25.777060 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:25.778284 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:25.778313 (+ 29us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:25.778607 (+ 294us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:25.779370 (+ 763us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:29:25.779407 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:29:25.780913 (+ 1506us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:29:25.785228 (+ 4315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:29:25.785231 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:29:25.785234 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:29:25.785620 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:29:25.786061 (+ 441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:29:25.786064 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:29:25.786070 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:29:25.786271 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:29:25.787035 (+ 764us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:29:25.787042 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:29:25.787291 (+ 249us) client_negotiation.cc:769] Sending connection context
0806 09:29:25.787426 (+ 135us) client_negotiation.cc:240] Negotiation successful
0806 09:29:25.787751 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":562,"thread_start_us":369,"threads_started":1}
I20260806 09:29:25.788528 352 heartbeater.cc:464] Registering TS with master...
I20260806 09:29:25.789005 32241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:25.771796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.3:32989 (local address 127.20.113.62:45237)
0806 09:29:25.772033 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:25.772042 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:25.772652 (+ 610us) server_negotiation.cc:407] Connection header received
0806 09:29:25.772881 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:25.772890 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:25.772992 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:25.773101 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:25.774666 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:25.775560 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:25.777221 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:25.777999 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:25.778159 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:25.781200 (+ 3041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:29:25.781208 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:29:25.781216 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:29:25.781267 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:29:25.785104 (+ 3837us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:25.785768 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:25.785774 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:25.785777 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:25.785927 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:25.786418 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:25.786426 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:25.786431 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:25.786854 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:29:25.788419 (+ 1565us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:25.788536 (+ 117us) server_negotiation.cc:299] Negotiation successful
0806 09:29:25.788773 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260806 09:29:25.789289 352 heartbeater.cc:511] Master 127.20.113.62:45237 requested a full tablet report, sending...
I20260806 09:29:25.790707 32095 ts_manager.cc:194] Registered new tserver with Master: d6930c871eec4d2f9c6efa9c8e742cba (127.20.113.3:38869)
I20260806 09:29:25.791729 32095 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.20.113.3@KRBTEST.COM'} at 127.20.113.3:53321
I20260806 09:29:25.794242 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 32631
I20260806 09:29:25.794438 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786008495763054-20932-0/minicluster-data/ts-2/wal/instance
I20260806 09:29:25.794885 20932 external_mini_cluster.cc:1428] Setting key ca6024d46e71b9e6b99cd7be588a30c6
I20260806 09:29:25.797629 20932 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 09:29:25 dist-test-slave-45dx krb5kdc[32029](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008565, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 09:29:25 dist-test-slave-45dx krb5kdc[32029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.20.113.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 09:29:25 dist-test-slave-45dx krb5kdc[32029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.20.113.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 09:29:26.260671 32627 heartbeater.cc:503] Master 127.20.113.62:45237 was elected leader, sending a full tablet report...
I20260806 09:29:26.794060 352 heartbeater.cc:503] Master 127.20.113.62:45237 was elected leader, sending a full tablet report...
W20260806 09:29:26.959750 367 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.104s user 0.000s sys 0.022s
W20260806 09:29:26.961073 32241 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 09:29:25.850140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53432 (local address 127.20.113.62:45237)
0806 09:29:25.850370 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:25.850376 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:25.850537 (+ 161us) server_negotiation.cc:407] Connection header received
0806 09:29:25.850753 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:25.850758 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:25.850840 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:25.850949 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:25.852344 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:25.853086 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:25.854408 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:25.854841 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:25.854981 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:26.960805 (+1105824us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53432: BlockingRecv error: recv got EOF from 127.0.0.1:53432 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
Aug 06 09:29:26 dist-test-slave-45dx krb5kdc[32029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008565, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM
I20260806 09:29:26.984933 32241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:26.965313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53436 (local address 127.20.113.62:45237)
0806 09:29:26.965469 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:26.965473 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:26.965872 (+ 399us) server_negotiation.cc:407] Connection header received
0806 09:29:26.966022 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:26.966026 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:26.966114 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:26.966197 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:26.967424 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:26.968226 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:26.969388 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:26.969786 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:26.969878 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:26.979917 (+ 10039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:29:26.979922 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:29:26.979925 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:29:26.979960 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:29:26.982702 (+ 2742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:26.983308 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:26.983312 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:26.983315 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:26.983444 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:26.983783 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:26.983786 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:26.983788 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:26.984046 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:29:26.984283 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:26.984521 (+ 238us) server_negotiation.cc:299] Negotiation successful
0806 09:29:26.984702 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 09:29:26.988852 32095 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53436:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 09:29:26.995874 32095 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 09:29:27.028131 386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:27.007800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.1:33839 (local address 127.0.0.1:47208)
0806 09:29:27.009754 (+ 1954us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:27.009799 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:27.010026 (+ 227us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:27.011114 (+ 1088us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:27.011121 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:27.011151 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:29:27.011598 (+ 447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:27.011611 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:27.014117 (+ 2506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:27.014125 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:27.024981 (+ 10856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:27.024996 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:27.027185 (+ 2189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:27.027191 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:27.027422 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:27.027469 (+ 47us) client_negotiation.cc:769] Sending connection context
0806 09:29:27.027592 (+ 123us) client_negotiation.cc:240] Negotiation successful
0806 09:29:27.027765 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1647,"thread_start_us":1458,"threads_started":1}
I20260806 09:29:27.030150 387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:27.007996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47208 (local address 127.20.113.1:33839)
0806 09:29:27.009279 (+ 1283us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:27.009290 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:27.010248 (+ 958us) server_negotiation.cc:407] Connection header received
0806 09:29:27.010372 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:27.010378 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:27.010657 (+ 279us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:27.010859 (+ 202us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:29:27.011800 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:27.013915 (+ 2115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:27.025198 (+ 11283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:27.026721 (+ 1523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:27.026852 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:27.027050 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:27.028722 (+ 1672us) server_negotiation.cc:299] Negotiation successful
0806 09:29:27.028894 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1103,"thread_start_us":968,"threads_started":1}
I20260806 09:29:27.034826 389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:27.008562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.2:45517 (local address 127.0.0.1:52408)
0806 09:29:27.022661 (+ 14099us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:27.022680 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:27.022780 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:27.025839 (+ 3059us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:27.025847 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:27.025882 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:29:27.026390 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:27.026404 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:27.028546 (+ 2142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:27.028554 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:27.031156 (+ 2602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:27.031180 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:27.033973 (+ 2793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:27.033984 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:27.034189 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:27.034223 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 09:29:27.034318 (+ 95us) client_negotiation.cc:240] Negotiation successful
0806 09:29:27.034465 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13978,"thread_start_us":14272,"threads_started":1}
I20260806 09:29:27.035531 391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:27.009294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52408 (local address 127.20.113.2:45517)
0806 09:29:27.024640 (+ 15346us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:27.024652 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:27.024689 (+ 37us) server_negotiation.cc:407] Connection header received
0806 09:29:27.024812 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:27.024823 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:27.025361 (+ 538us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:27.025589 (+ 228us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:29:27.026611 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:27.028347 (+ 1736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:27.031403 (+ 3056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:27.033392 (+ 1989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:27.033531 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:27.033757 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:27.035052 (+ 1295us) server_negotiation.cc:299] Negotiation successful
0806 09:29:27.035231 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15141,"thread_start_us":15626,"threads_started":1}
I20260806 09:29:27.038093 32308 tablet_service.cc:1467] Processing CreateTablet for tablet 5e854bf287754acfbc5a11a791dc4650 (DEFAULT_TABLE table=test-table [id=6a7486f01a414f0eae4dc3248f815707]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:29:27.039441 386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:27.010177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.3:38869 (local address 127.0.0.1:50648)
0806 09:29:27.028469 (+ 18292us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:27.028486 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:27.028569 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:27.031323 (+ 2754us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:27.031327 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:27.031350 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:29:27.031537 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:27.031547 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:27.033838 (+ 2291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:27.033844 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:27.035771 (+ 1927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:27.035785 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:27.038689 (+ 2904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:27.038695 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:27.038898 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:27.038929 (+ 31us) client_negotiation.cc:769] Sending connection context
0806 09:29:27.038988 (+ 59us) client_negotiation.cc:240] Negotiation successful
0806 09:29:27.039116 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18196,"thread_start_us":23446,"threads_started":1}
I20260806 09:29:27.039383 32518 tablet_service.cc:1467] Processing CreateTablet for tablet 5e854bf287754acfbc5a11a791dc4650 (DEFAULT_TABLE table=test-table [id=6a7486f01a414f0eae4dc3248f815707]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:29:27.040030 32308 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e854bf287754acfbc5a11a791dc4650. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:29:27.040256 394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:27.010787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50648 (local address 127.20.113.3:38869)
0806 09:29:27.030808 (+ 20021us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:27.030816 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:27.030842 (+ 26us) server_negotiation.cc:407] Connection header received
0806 09:29:27.030926 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:27.030933 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:27.031131 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:27.031697 (+ 566us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:29:27.031919 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:27.033398 (+ 1479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:27.036001 (+ 2603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:27.038497 (+ 2496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:27.039447 (+ 950us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:27.039701 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:27.039775 (+ 74us) server_negotiation.cc:299] Negotiation successful
0806 09:29:27.039960 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19871,"thread_start_us":19745,"threads_started":1}
I20260806 09:29:27.040815 32518 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e854bf287754acfbc5a11a791dc4650. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:29:27.043033 32721 tablet_service.cc:1467] Processing CreateTablet for tablet 5e854bf287754acfbc5a11a791dc4650 (DEFAULT_TABLE table=test-table [id=6a7486f01a414f0eae4dc3248f815707]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:29:27.044088 32721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e854bf287754acfbc5a11a791dc4650. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:29:27.054171 395 tablet_bootstrap.cc:492] T 5e854bf287754acfbc5a11a791dc4650 P d6930c871eec4d2f9c6efa9c8e742cba: Bootstrap starting.
I20260806 09:29:27.055176 397 tablet_bootstrap.cc:492] T 5e854bf287754acfbc5a11a791dc4650 P a27e10c55c654384ae155607041dad72: Bootstrap starting.
I20260806 09:29:27.055968 396 tablet_bootstrap.cc:492] T 5e854bf287754acfbc5a11a791dc4650 P d9485c4273194777b6ccf02c3e050e6f: Bootstrap starting.
I20260806 09:29:27.059414 395 tablet_bootstrap.cc:654] T 5e854bf287754acfbc5a11a791dc4650 P d6930c871eec4d2f9c6efa9c8e742cba: Neither blocks nor log segments found. Creating new log.
I20260806 09:29:27.059594 397 tablet_bootstrap.cc:654] T 5e854bf287754acfbc5a11a791dc4650 P a27e10c55c654384ae155607041dad72: Neither blocks nor log segments found. Creating new log.
I20260806 09:29:27.059592 396 tablet_bootstrap.cc:654] T 5e854bf287754acfbc5a11a791dc4650 P d9485c4273194777b6ccf02c3e050e6f: Neither blocks nor log segments found. Creating new log.
I20260806 09:29:27.060585 395 log.cc:824] T 5e854bf287754acfbc5a11a791dc4650 P d6930c871eec4d2f9c6efa9c8e742cba: Log is configured to *not* fsync() on all Append() calls
I20260806 09:29:27.060876 396 log.cc:824] T 5e854bf287754acfbc5a11a791dc4650 P d9485c4273194777b6ccf02c3e050e6f: Log is configured to *not* fsync() on all Append() calls
I20260806 09:29:27.060877 397 log.cc:824] T 5e854bf287754acfbc5a11a791dc4650 P a27e10c55c654384ae155607041dad72: Log is configured to *not* fsync() on all Append() calls
I20260806 09:29:27.063812 396 tablet_bootstrap.cc:492] T 5e854bf287754acfbc5a11a791dc4650 P d9485c4273194777b6ccf02c3e050e6f: No bootstrap required, opened a new log
I20260806 09:29:27.063817 397 tablet_bootstrap.cc:492] T 5e854bf287754acfbc5a11a791dc4650 P a27e10c55c654384ae155607041dad72: No bootstrap required, opened a new log
I20260806 09:29:27.064126 397 ts_tablet_manager.cc:1397] T 5e854bf287754acfbc5a11a791dc4650 P a27e10c55c654384ae155607041dad72: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260806 09:29:27.064157 396 ts_tablet_manager.cc:1397] T 5e854bf287754acfbc5a11a791dc4650 P d9485c4273194777b6ccf02c3e050e6f: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260806 09:29:27.066051 395 tablet_bootstrap.cc:492] T 5e854bf287754acfbc5a11a791dc4650 P d6930c871eec4d2f9c6efa9c8e742cba: No bootstrap required, opened a new log
I20260806 09:29:27.066332 395 ts_tablet_manager.cc:1397] T 5e854bf287754acfbc5a11a791dc4650 P d6930c871eec4d2f9c6efa9c8e742cba: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.000s
I20260806 09:29:27.071862 397 raft_consensus.cc:348] T 5e854bf287754acfbc5a11a791dc4650 P a27e10c55c654384ae155607041dad72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a27e10c55c654384ae155607041dad72" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33839 } } peers { permanent_uuid: "d6930c871eec4d2f9c6efa9c8e742cba" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 38869 } } peers { permanent_uuid: "d9485c4273194777b6ccf02c3e050e6f" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 45517 } }
I20260806 09:29:27.072225 397 raft_consensus.cc:374] T 5e854bf287754acfbc5a11a791dc4650 P a27e10c55c654384ae155607041dad72 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:29:27.072386 397 raft_consensus.cc:729] T 5e854bf287754acfbc5a11a791dc4650 P a27e10c55c654384ae155607041dad72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a27e10c55c654384ae155607041dad72, State: Initialized, Role: FOLLOWER
I20260806 09:29:27.072984 397 consensus_queue.cc:260] T 5e854bf287754acfbc5a11a791dc4650 P a27e10c55c654384ae155607041dad72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a27e10c55c654384ae155607041dad72" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33839 } } peers { permanent_uuid: "d6930c871eec4d2f9c6efa9c8e742cba" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 38869 } } peers { permanent_uuid: "d9485c4273194777b6ccf02c3e050e6f" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 45517 } }
I20260806 09:29:27.074723 397 ts_tablet_manager.cc:1428] T 5e854bf287754acfbc5a11a791dc4650 P a27e10c55c654384ae155607041dad72: Time spent starting tablet: real 0.010s user 0.009s sys 0.000s
I20260806 09:29:27.074879 396 raft_consensus.cc:348] T 5e854bf287754acfbc5a11a791dc4650 P d9485c4273194777b6ccf02c3e050e6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a27e10c55c654384ae155607041dad72" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33839 } } peers { permanent_uuid: "d6930c871eec4d2f9c6efa9c8e742cba" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 38869 } } peers { permanent_uuid: "d9485c4273194777b6ccf02c3e050e6f" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 45517 } }
I20260806 09:29:27.075348 396 raft_consensus.cc:374] T 5e854bf287754acfbc5a11a791dc4650 P d9485c4273194777b6ccf02c3e050e6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:29:27.075459 396 raft_consensus.cc:729] T 5e854bf287754acfbc5a11a791dc4650 P d9485c4273194777b6ccf02c3e050e6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9485c4273194777b6ccf02c3e050e6f, State: Initialized, Role: FOLLOWER
I20260806 09:29:27.076097 396 consensus_queue.cc:260] T 5e854bf287754acfbc5a11a791dc4650 P d9485c4273194777b6ccf02c3e050e6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a27e10c55c654384ae155607041dad72" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33839 } } peers { permanent_uuid: "d6930c871eec4d2f9c6efa9c8e742cba" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 38869 } } peers { permanent_uuid: "d9485c4273194777b6ccf02c3e050e6f" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 45517 } }
I20260806 09:29:27.076812 395 raft_consensus.cc:348] T 5e854bf287754acfbc5a11a791dc4650 P d6930c871eec4d2f9c6efa9c8e742cba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a27e10c55c654384ae155607041dad72" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33839 } } peers { permanent_uuid: "d6930c871eec4d2f9c6efa9c8e742cba" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 38869 } } peers { permanent_uuid: "d9485c4273194777b6ccf02c3e050e6f" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 45517 } }
I20260806 09:29:27.077315 395 raft_consensus.cc:374] T 5e854bf287754acfbc5a11a791dc4650 P d6930c871eec4d2f9c6efa9c8e742cba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:29:27.077445 395 raft_consensus.cc:729] T 5e854bf287754acfbc5a11a791dc4650 P d6930c871eec4d2f9c6efa9c8e742cba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6930c871eec4d2f9c6efa9c8e742cba, State: Initialized, Role: FOLLOWER
I20260806 09:29:27.078004 396 ts_tablet_manager.cc:1428] T 5e854bf287754acfbc5a11a791dc4650 P d9485c4273194777b6ccf02c3e050e6f: Time spent starting tablet: real 0.014s user 0.012s sys 0.000s
I20260806 09:29:27.078230 395 consensus_queue.cc:260] T 5e854bf287754acfbc5a11a791dc4650 P d6930c871eec4d2f9c6efa9c8e742cba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a27e10c55c654384ae155607041dad72" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33839 } } peers { permanent_uuid: "d6930c871eec4d2f9c6efa9c8e742cba" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 38869 } } peers { permanent_uuid: "d9485c4273194777b6ccf02c3e050e6f" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 45517 } }
I20260806 09:29:27.080035 395 ts_tablet_manager.cc:1428] T 5e854bf287754acfbc5a11a791dc4650 P d6930c871eec4d2f9c6efa9c8e742cba: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260806 09:29:27.149802 403 raft_consensus.cc:482] T 5e854bf287754acfbc5a11a791dc4650 P a27e10c55c654384ae155607041dad72 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 09:29:27.150173 403 raft_consensus.cc:504] T 5e854bf287754acfbc5a11a791dc4650 P a27e10c55c654384ae155607041dad72 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a27e10c55c654384ae155607041dad72" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33839 } } peers { permanent_uuid: "d6930c871eec4d2f9c6efa9c8e742cba" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 38869 } } peers { permanent_uuid: "d9485c4273194777b6ccf02c3e050e6f" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 45517 } }
I20260806 09:29:27.151849 403 leader_election.cc:288] T 5e854bf287754acfbc5a11a791dc4650 P a27e10c55c654384ae155607041dad72 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6930c871eec4d2f9c6efa9c8e742cba (127.20.113.3:38869), d9485c4273194777b6ccf02c3e050e6f (127.20.113.2:45517)
I20260806 09:29:27.159629 394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:27.152434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.1:60631 (local address 127.20.113.3:38869)
0806 09:29:27.152620 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:27.152625 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:27.153394 (+ 769us) server_negotiation.cc:407] Connection header received
0806 09:29:27.153629 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:27.153635 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:27.153721 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:27.153828 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:29:27.154409 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:27.155131 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:27.157323 (+ 2192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:27.158612 (+ 1289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:27.158783 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:27.158862 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:27.159262 (+ 400us) server_negotiation.cc:299] Negotiation successful
0806 09:29:27.159394 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 09:29:27.159739 406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:27.152536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.3:38869 (local address 127.20.113.1:60631)
0806 09:29:27.153271 (+ 735us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:27.153289 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:27.153470 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:27.153852 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:27.153856 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:27.153883 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:29:27.154237 (+ 354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:27.154245 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:27.155265 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:27.155270 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:27.157059 (+ 1789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:27.157075 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:27.158851 (+ 1776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:27.158855 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:27.159054 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:27.159097 (+ 43us) client_negotiation.cc:769] Sending connection context
0806 09:29:27.159192 (+ 95us) client_negotiation.cc:240] Negotiation successful
0806 09:29:27.159318 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":624,"mutex_wait_us":1,"thread_start_us":442,"threads_started":1}
I20260806 09:29:27.159786 391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:27.152434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.1:42299 (local address 127.20.113.2:45517)
0806 09:29:27.152661 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:27.152682 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:27.153883 (+ 1201us) server_negotiation.cc:407] Connection header received
0806 09:29:27.154117 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:27.154122 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:27.154187 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:27.154283 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:29:27.154856 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:27.155557 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:27.157638 (+ 2081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:27.158869 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:27.159037 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:27.159127 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:27.159431 (+ 304us) server_negotiation.cc:299] Negotiation successful
0806 09:29:27.159552 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 09:29:27.160077 408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:27.152836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.2:45517 (local address 127.20.113.1:42299)
0806 09:29:27.153771 (+ 935us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:27.153786 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:27.153976 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:27.154300 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:27.154304 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:27.154332 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:29:27.154693 (+ 361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:27.154702 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:27.155714 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:27.155719 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:27.157390 (+ 1671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:27.157404 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:27.159093 (+ 1689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:27.159098 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:27.159281 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:27.159310 (+ 29us) client_negotiation.cc:769] Sending connection context
0806 09:29:27.159390 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 09:29:27.159592 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":788,"mutex_wait_us":205,"thread_start_us":560,"threads_started":1}
I20260806 09:29:27.160895 32748 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e854bf287754acfbc5a11a791dc4650" candidate_uuid: "a27e10c55c654384ae155607041dad72" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6930c871eec4d2f9c6efa9c8e742cba" is_pre_election: true
I20260806 09:29:27.160974 32551 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e854bf287754acfbc5a11a791dc4650" candidate_uuid: "a27e10c55c654384ae155607041dad72" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9485c4273194777b6ccf02c3e050e6f" is_pre_election: true
I20260806 09:29:27.161510 32748 raft_consensus.cc:2454] T 5e854bf287754acfbc5a11a791dc4650 P d6930c871eec4d2f9c6efa9c8e742cba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a27e10c55c654384ae155607041dad72 in term 0.
I20260806 09:29:27.161574 32551 raft_consensus.cc:2454] T 5e854bf287754acfbc5a11a791dc4650 P d9485c4273194777b6ccf02c3e050e6f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a27e10c55c654384ae155607041dad72 in term 0.
I20260806 09:29:27.162215 32230 leader_election.cc:302] T 5e854bf287754acfbc5a11a791dc4650 P a27e10c55c654384ae155607041dad72 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a27e10c55c654384ae155607041dad72, d6930c871eec4d2f9c6efa9c8e742cba; no voters:
I20260806 09:29:27.162649 403 raft_consensus.cc:2781] T 5e854bf287754acfbc5a11a791dc4650 P a27e10c55c654384ae155607041dad72 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 09:29:27.162811 403 raft_consensus.cc:482] T 5e854bf287754acfbc5a11a791dc4650 P a27e10c55c654384ae155607041dad72 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 09:29:27.162932 403 raft_consensus.cc:3037] T 5e854bf287754acfbc5a11a791dc4650 P a27e10c55c654384ae155607041dad72 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:29:27.165393 403 raft_consensus.cc:504] T 5e854bf287754acfbc5a11a791dc4650 P a27e10c55c654384ae155607041dad72 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a27e10c55c654384ae155607041dad72" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33839 } } peers { permanent_uuid: "d6930c871eec4d2f9c6efa9c8e742cba" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 38869 } } peers { permanent_uuid: "d9485c4273194777b6ccf02c3e050e6f" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 45517 } }
I20260806 09:29:27.166030 403 leader_election.cc:288] T 5e854bf287754acfbc5a11a791dc4650 P a27e10c55c654384ae155607041dad72 [CANDIDATE]: Term 1 election: Requested vote from peers d6930c871eec4d2f9c6efa9c8e742cba (127.20.113.3:38869), d9485c4273194777b6ccf02c3e050e6f (127.20.113.2:45517)
I20260806 09:29:27.166404 32748 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e854bf287754acfbc5a11a791dc4650" candidate_uuid: "a27e10c55c654384ae155607041dad72" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6930c871eec4d2f9c6efa9c8e742cba"
I20260806 09:29:27.166625 32748 raft_consensus.cc:3037] T 5e854bf287754acfbc5a11a791dc4650 P d6930c871eec4d2f9c6efa9c8e742cba [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:29:27.166630 32551 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e854bf287754acfbc5a11a791dc4650" candidate_uuid: "a27e10c55c654384ae155607041dad72" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9485c4273194777b6ccf02c3e050e6f"
I20260806 09:29:27.166842 32551 raft_consensus.cc:3037] T 5e854bf287754acfbc5a11a791dc4650 P d9485c4273194777b6ccf02c3e050e6f [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:29:27.168883 32748 raft_consensus.cc:2454] T 5e854bf287754acfbc5a11a791dc4650 P d6930c871eec4d2f9c6efa9c8e742cba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a27e10c55c654384ae155607041dad72 in term 1.
I20260806 09:29:27.169219 32551 raft_consensus.cc:2454] T 5e854bf287754acfbc5a11a791dc4650 P d9485c4273194777b6ccf02c3e050e6f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a27e10c55c654384ae155607041dad72 in term 1.
I20260806 09:29:27.169251 32230 leader_election.cc:302] T 5e854bf287754acfbc5a11a791dc4650 P a27e10c55c654384ae155607041dad72 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a27e10c55c654384ae155607041dad72, d6930c871eec4d2f9c6efa9c8e742cba; no voters:
I20260806 09:29:27.169520 403 raft_consensus.cc:2781] T 5e854bf287754acfbc5a11a791dc4650 P a27e10c55c654384ae155607041dad72 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:29:27.169956 403 raft_consensus.cc:686] T 5e854bf287754acfbc5a11a791dc4650 P a27e10c55c654384ae155607041dad72 [term 1 LEADER]: Becoming Leader. State: Replica: a27e10c55c654384ae155607041dad72, State: Running, Role: LEADER
I20260806 09:29:27.170406 403 consensus_queue.cc:237] T 5e854bf287754acfbc5a11a791dc4650 P a27e10c55c654384ae155607041dad72 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a27e10c55c654384ae155607041dad72" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33839 } } peers { permanent_uuid: "d6930c871eec4d2f9c6efa9c8e742cba" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 38869 } } peers { permanent_uuid: "d9485c4273194777b6ccf02c3e050e6f" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 45517 } }
I20260806 09:29:27.174789 32092 catalog_manager.cc:5495] T 5e854bf287754acfbc5a11a791dc4650 P a27e10c55c654384ae155607041dad72 reported cstate change: term changed from 0 to 1, leader changed from <none> to a27e10c55c654384ae155607041dad72 (127.20.113.1). New cstate: current_term: 1 leader_uuid: "a27e10c55c654384ae155607041dad72" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a27e10c55c654384ae155607041dad72" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33839 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d6930c871eec4d2f9c6efa9c8e742cba" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 38869 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9485c4273194777b6ccf02c3e050e6f" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 45517 } health_report { overall_health: UNKNOWN } } }
W20260806 09:29:27.202828 32411 tablet.cc:2369] T 5e854bf287754acfbc5a11a791dc4650 P a27e10c55c654384ae155607041dad72: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 09:29:27.207428 387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:27.202080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47216 (local address 127.20.113.1:33839)
0806 09:29:27.202284 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:27.202290 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:27.202332 (+ 42us) server_negotiation.cc:407] Connection header received
0806 09:29:27.202608 (+ 276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:27.202612 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:27.202697 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:27.202859 (+ 162us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 09:29:27.203215 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:27.203919 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:27.205324 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:27.205731 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:27.205866 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:27.206336 (+ 470us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 09:29:27.206815 (+ 479us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 09:29:27.206980 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:27.207085 (+ 105us) server_negotiation.cc:299] Negotiation successful
0806 09:29:27.207164 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 09:29:27.215291 32748 raft_consensus.cc:1261] T 5e854bf287754acfbc5a11a791dc4650 P d6930c871eec4d2f9c6efa9c8e742cba [term 1 FOLLOWER]: Refusing update from remote peer a27e10c55c654384ae155607041dad72: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 09:29:27.215291 32551 raft_consensus.cc:1261] T 5e854bf287754acfbc5a11a791dc4650 P d9485c4273194777b6ccf02c3e050e6f [term 1 FOLLOWER]: Refusing update from remote peer a27e10c55c654384ae155607041dad72: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 09:29:27.216439 409 consensus_queue.cc:1035] T 5e854bf287754acfbc5a11a791dc4650 P a27e10c55c654384ae155607041dad72 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6930c871eec4d2f9c6efa9c8e742cba" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 38869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 09:29:27.216835 403 consensus_queue.cc:1035] T 5e854bf287754acfbc5a11a791dc4650 P a27e10c55c654384ae155607041dad72 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9485c4273194777b6ccf02c3e050e6f" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 45517 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 09:29:27.228716 414 mvcc.cc:205] Tried to move back new op lower bound from 7315491091304357888 to 7315491091134005248. Current Snapshot: MvccSnapshot[applied={T|T < 7315491091304357888}]
I20260806 09:29:27.244077 32092 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53436:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 09:29:27.244434 32092 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53436:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 09:29:27.248361 32092 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 81679e1de52c40d883ea9f9173070aa7: Sending DeleteTablet for 3 replicas of tablet 5e854bf287754acfbc5a11a791dc4650
I20260806 09:29:27.249650 32308 tablet_service.cc:1514] Processing DeleteTablet for tablet 5e854bf287754acfbc5a11a791dc4650 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:29:27 UTC) from {username='oryx', principal='oryx/127.20.113.62@KRBTEST.COM'} at 127.0.0.1:47208
I20260806 09:29:27.249655 32518 tablet_service.cc:1514] Processing DeleteTablet for tablet 5e854bf287754acfbc5a11a791dc4650 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:29:27 UTC) from {username='oryx', principal='oryx/127.20.113.62@KRBTEST.COM'} at 127.0.0.1:52408
I20260806 09:29:27.250465 32721 tablet_service.cc:1514] Processing DeleteTablet for tablet 5e854bf287754acfbc5a11a791dc4650 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:29:27 UTC) from {username='oryx', principal='oryx/127.20.113.62@KRBTEST.COM'} at 127.0.0.1:50648
I20260806 09:29:27.250676 32092 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53436:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 09:29:27.251252 32092 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 09:29:27.259675 430 tablet_replica.cc:330] T 5e854bf287754acfbc5a11a791dc4650 P d6930c871eec4d2f9c6efa9c8e742cba: stopping tablet replica
I20260806 09:29:27.260949 430 raft_consensus.cc:2229] T 5e854bf287754acfbc5a11a791dc4650 P d6930c871eec4d2f9c6efa9c8e742cba [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 09:29:27.262343 430 raft_consensus.cc:2258] T 5e854bf287754acfbc5a11a791dc4650 P d6930c871eec4d2f9c6efa9c8e742cba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 09:29:27.264775 32517 tablet_service.cc:1467] Processing CreateTablet for tablet 2c23e01ea61e43c6b701cab05f3d6a8a (DEFAULT_TABLE table=test-table [id=60af8cd5737540b0afbbadc4b847c9cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:29:27.265358 32517 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c23e01ea61e43c6b701cab05f3d6a8a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:29:27.266260 430 ts_tablet_manager.cc:1905] T 5e854bf287754acfbc5a11a791dc4650 P d6930c871eec4d2f9c6efa9c8e742cba: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 09:29:27.269812 32305 tablet_service.cc:1467] Processing CreateTablet for tablet 2c23e01ea61e43c6b701cab05f3d6a8a (DEFAULT_TABLE table=test-table [id=60af8cd5737540b0afbbadc4b847c9cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:29:27.270771 32721 tablet_service.cc:1467] Processing CreateTablet for tablet 2c23e01ea61e43c6b701cab05f3d6a8a (DEFAULT_TABLE table=test-table [id=60af8cd5737540b0afbbadc4b847c9cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:29:27.271265 32721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c23e01ea61e43c6b701cab05f3d6a8a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:29:27.274163 396 tablet_bootstrap.cc:492] T 2c23e01ea61e43c6b701cab05f3d6a8a P d9485c4273194777b6ccf02c3e050e6f: Bootstrap starting.
I20260806 09:29:27.275918 32305 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c23e01ea61e43c6b701cab05f3d6a8a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:29:27.276629 396 tablet_bootstrap.cc:654] T 2c23e01ea61e43c6b701cab05f3d6a8a P d9485c4273194777b6ccf02c3e050e6f: Neither blocks nor log segments found. Creating new log.
I20260806 09:29:27.277592 395 tablet_bootstrap.cc:492] T 2c23e01ea61e43c6b701cab05f3d6a8a P d6930c871eec4d2f9c6efa9c8e742cba: Bootstrap starting.
I20260806 09:29:27.278672 430 ts_tablet_manager.cc:1918] T 5e854bf287754acfbc5a11a791dc4650 P d6930c871eec4d2f9c6efa9c8e742cba: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 09:29:27.278968 430 log.cc:1196] T 5e854bf287754acfbc5a11a791dc4650 P d6930c871eec4d2f9c6efa9c8e742cba: Deleting WAL directory at /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786008495763054-20932-0/minicluster-data/ts-2/wal/wals/5e854bf287754acfbc5a11a791dc4650
I20260806 09:29:27.279350 395 tablet_bootstrap.cc:654] T 2c23e01ea61e43c6b701cab05f3d6a8a P d6930c871eec4d2f9c6efa9c8e742cba: Neither blocks nor log segments found. Creating new log.
I20260806 09:29:27.279601 430 ts_tablet_manager.cc:1939] T 5e854bf287754acfbc5a11a791dc4650 P d6930c871eec4d2f9c6efa9c8e742cba: Deleting consensus metadata
I20260806 09:29:27.280359 396 tablet_bootstrap.cc:492] T 2c23e01ea61e43c6b701cab05f3d6a8a P d9485c4273194777b6ccf02c3e050e6f: No bootstrap required, opened a new log
I20260806 09:29:27.280543 396 ts_tablet_manager.cc:1397] T 2c23e01ea61e43c6b701cab05f3d6a8a P d9485c4273194777b6ccf02c3e050e6f: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.001s
I20260806 09:29:27.281371 32075 catalog_manager.cc:4841] TS d6930c871eec4d2f9c6efa9c8e742cba (127.20.113.3:38869): tablet 5e854bf287754acfbc5a11a791dc4650 (table test-table [id=6a7486f01a414f0eae4dc3248f815707]) successfully deleted
I20260806 09:29:27.281683 396 raft_consensus.cc:348] T 2c23e01ea61e43c6b701cab05f3d6a8a P d9485c4273194777b6ccf02c3e050e6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a27e10c55c654384ae155607041dad72" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33839 } } peers { permanent_uuid: "d9485c4273194777b6ccf02c3e050e6f" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 45517 } } peers { permanent_uuid: "d6930c871eec4d2f9c6efa9c8e742cba" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 38869 } }
I20260806 09:29:27.282029 396 raft_consensus.cc:374] T 2c23e01ea61e43c6b701cab05f3d6a8a P d9485c4273194777b6ccf02c3e050e6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:29:27.282145 396 raft_consensus.cc:729] T 2c23e01ea61e43c6b701cab05f3d6a8a P d9485c4273194777b6ccf02c3e050e6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9485c4273194777b6ccf02c3e050e6f, State: Initialized, Role: FOLLOWER
I20260806 09:29:27.283880 396 consensus_queue.cc:260] T 2c23e01ea61e43c6b701cab05f3d6a8a P d9485c4273194777b6ccf02c3e050e6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a27e10c55c654384ae155607041dad72" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33839 } } peers { permanent_uuid: "d9485c4273194777b6ccf02c3e050e6f" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 45517 } } peers { permanent_uuid: "d6930c871eec4d2f9c6efa9c8e742cba" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 38869 } }
I20260806 09:29:27.285082 396 ts_tablet_manager.cc:1428] T 2c23e01ea61e43c6b701cab05f3d6a8a P d9485c4273194777b6ccf02c3e050e6f: Time spent starting tablet: real 0.004s user 0.000s sys 0.003s
I20260806 09:29:27.291412 395 tablet_bootstrap.cc:492] T 2c23e01ea61e43c6b701cab05f3d6a8a P d6930c871eec4d2f9c6efa9c8e742cba: No bootstrap required, opened a new log
I20260806 09:29:27.292156 395 ts_tablet_manager.cc:1397] T 2c23e01ea61e43c6b701cab05f3d6a8a P d6930c871eec4d2f9c6efa9c8e742cba: Time spent bootstrapping tablet: real 0.015s user 0.008s sys 0.005s
I20260806 09:29:27.293447 395 raft_consensus.cc:348] T 2c23e01ea61e43c6b701cab05f3d6a8a P d6930c871eec4d2f9c6efa9c8e742cba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a27e10c55c654384ae155607041dad72" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33839 } } peers { permanent_uuid: "d9485c4273194777b6ccf02c3e050e6f" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 45517 } } peers { permanent_uuid: "d6930c871eec4d2f9c6efa9c8e742cba" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 38869 } }
I20260806 09:29:27.293829 395 raft_consensus.cc:374] T 2c23e01ea61e43c6b701cab05f3d6a8a P d6930c871eec4d2f9c6efa9c8e742cba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:29:27.294031 395 raft_consensus.cc:729] T 2c23e01ea61e43c6b701cab05f3d6a8a P d6930c871eec4d2f9c6efa9c8e742cba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6930c871eec4d2f9c6efa9c8e742cba, State: Initialized, Role: FOLLOWER
I20260806 09:29:27.294476 395 consensus_queue.cc:260] T 2c23e01ea61e43c6b701cab05f3d6a8a P d6930c871eec4d2f9c6efa9c8e742cba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a27e10c55c654384ae155607041dad72" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33839 } } peers { permanent_uuid: "d9485c4273194777b6ccf02c3e050e6f" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 45517 } } peers { permanent_uuid: "d6930c871eec4d2f9c6efa9c8e742cba" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 38869 } }
I20260806 09:29:27.296351 397 tablet_bootstrap.cc:492] T 2c23e01ea61e43c6b701cab05f3d6a8a P a27e10c55c654384ae155607041dad72: Bootstrap starting.
I20260806 09:29:27.297164 427 tablet_replica.cc:330] T 5e854bf287754acfbc5a11a791dc4650 P a27e10c55c654384ae155607041dad72: stopping tablet replica
I20260806 09:29:27.297652 427 raft_consensus.cc:2229] T 5e854bf287754acfbc5a11a791dc4650 P a27e10c55c654384ae155607041dad72 [term 1 LEADER]: Raft consensus shutting down.
I20260806 09:29:27.298017 395 ts_tablet_manager.cc:1428] T 2c23e01ea61e43c6b701cab05f3d6a8a P d6930c871eec4d2f9c6efa9c8e742cba: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260806 09:29:27.298179 397 tablet_bootstrap.cc:654] T 2c23e01ea61e43c6b701cab05f3d6a8a P a27e10c55c654384ae155607041dad72: Neither blocks nor log segments found. Creating new log.
I20260806 09:29:27.298352 427 raft_consensus.cc:2258] T 5e854bf287754acfbc5a11a791dc4650 P a27e10c55c654384ae155607041dad72 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 09:29:27.299081 426 tablet_replica.cc:330] T 5e854bf287754acfbc5a11a791dc4650 P d9485c4273194777b6ccf02c3e050e6f: stopping tablet replica
I20260806 09:29:27.301430 427 ts_tablet_manager.cc:1905] T 5e854bf287754acfbc5a11a791dc4650 P a27e10c55c654384ae155607041dad72: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 09:29:27.301501 397 tablet_bootstrap.cc:492] T 2c23e01ea61e43c6b701cab05f3d6a8a P a27e10c55c654384ae155607041dad72: No bootstrap required, opened a new log
I20260806 09:29:27.301621 397 ts_tablet_manager.cc:1397] T 2c23e01ea61e43c6b701cab05f3d6a8a P a27e10c55c654384ae155607041dad72: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260806 09:29:27.300781 426 raft_consensus.cc:2229] T 5e854bf287754acfbc5a11a791dc4650 P d9485c4273194777b6ccf02c3e050e6f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 09:29:27.302165 426 raft_consensus.cc:2258] T 5e854bf287754acfbc5a11a791dc4650 P d9485c4273194777b6ccf02c3e050e6f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 09:29:27.302736 397 raft_consensus.cc:348] T 2c23e01ea61e43c6b701cab05f3d6a8a P a27e10c55c654384ae155607041dad72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a27e10c55c654384ae155607041dad72" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33839 } } peers { permanent_uuid: "d9485c4273194777b6ccf02c3e050e6f" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 45517 } } peers { permanent_uuid: "d6930c871eec4d2f9c6efa9c8e742cba" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 38869 } }
I20260806 09:29:27.303043 397 raft_consensus.cc:374] T 2c23e01ea61e43c6b701cab05f3d6a8a P a27e10c55c654384ae155607041dad72 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:29:27.303191 397 raft_consensus.cc:729] T 2c23e01ea61e43c6b701cab05f3d6a8a P a27e10c55c654384ae155607041dad72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a27e10c55c654384ae155607041dad72, State: Initialized, Role: FOLLOWER
I20260806 09:29:27.303565 397 consensus_queue.cc:260] T 2c23e01ea61e43c6b701cab05f3d6a8a P a27e10c55c654384ae155607041dad72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a27e10c55c654384ae155607041dad72" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33839 } } peers { permanent_uuid: "d9485c4273194777b6ccf02c3e050e6f" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 45517 } } peers { permanent_uuid: "d6930c871eec4d2f9c6efa9c8e742cba" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 38869 } }
I20260806 09:29:27.304023 426 ts_tablet_manager.cc:1905] T 5e854bf287754acfbc5a11a791dc4650 P d9485c4273194777b6ccf02c3e050e6f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 09:29:27.306339 397 ts_tablet_manager.cc:1428] T 2c23e01ea61e43c6b701cab05f3d6a8a P a27e10c55c654384ae155607041dad72: Time spent starting tablet: real 0.005s user 0.000s sys 0.002s
I20260806 09:29:27.308818 427 ts_tablet_manager.cc:1918] T 5e854bf287754acfbc5a11a791dc4650 P a27e10c55c654384ae155607041dad72: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 09:29:27.309019 427 log.cc:1196] T 5e854bf287754acfbc5a11a791dc4650 P a27e10c55c654384ae155607041dad72: Deleting WAL directory at /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786008495763054-20932-0/minicluster-data/ts-0/wal/wals/5e854bf287754acfbc5a11a791dc4650
I20260806 09:29:27.309509 427 ts_tablet_manager.cc:1939] T 5e854bf287754acfbc5a11a791dc4650 P a27e10c55c654384ae155607041dad72: Deleting consensus metadata
I20260806 09:29:27.310835 32075 catalog_manager.cc:4841] TS a27e10c55c654384ae155607041dad72 (127.20.113.1:33839): tablet 5e854bf287754acfbc5a11a791dc4650 (table test-table [id=6a7486f01a414f0eae4dc3248f815707]) successfully deleted
I20260806 09:29:27.312139 426 ts_tablet_manager.cc:1918] T 5e854bf287754acfbc5a11a791dc4650 P d9485c4273194777b6ccf02c3e050e6f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 09:29:27.312348 426 log.cc:1196] T 5e854bf287754acfbc5a11a791dc4650 P d9485c4273194777b6ccf02c3e050e6f: Deleting WAL directory at /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786008495763054-20932-0/minicluster-data/ts-1/wal/wals/5e854bf287754acfbc5a11a791dc4650
I20260806 09:29:27.312947 426 ts_tablet_manager.cc:1939] T 5e854bf287754acfbc5a11a791dc4650 P d9485c4273194777b6ccf02c3e050e6f: Deleting consensus metadata
I20260806 09:29:27.314618 32077 catalog_manager.cc:4841] TS d9485c4273194777b6ccf02c3e050e6f (127.20.113.2:45517): tablet 5e854bf287754acfbc5a11a791dc4650 (table test-table [id=6a7486f01a414f0eae4dc3248f815707]) successfully deleted
I20260806 09:29:27.372216 405 raft_consensus.cc:482] T 2c23e01ea61e43c6b701cab05f3d6a8a P d6930c871eec4d2f9c6efa9c8e742cba [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 09:29:27.372539 405 raft_consensus.cc:504] T 2c23e01ea61e43c6b701cab05f3d6a8a P d6930c871eec4d2f9c6efa9c8e742cba [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a27e10c55c654384ae155607041dad72" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33839 } } peers { permanent_uuid: "d9485c4273194777b6ccf02c3e050e6f" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 45517 } } peers { permanent_uuid: "d6930c871eec4d2f9c6efa9c8e742cba" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 38869 } }
I20260806 09:29:27.374130 405 leader_election.cc:288] T 2c23e01ea61e43c6b701cab05f3d6a8a P d6930c871eec4d2f9c6efa9c8e742cba [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a27e10c55c654384ae155607041dad72 (127.20.113.1:33839), d9485c4273194777b6ccf02c3e050e6f (127.20.113.2:45517)
I20260806 09:29:27.382412 436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:27.374474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.1:33839 (local address 127.20.113.3:53177)
0806 09:29:27.375475 (+ 1001us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:27.375504 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:27.375776 (+ 272us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:27.376400 (+ 624us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:27.376404 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:27.376431 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:29:27.376966 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:27.376978 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:27.378616 (+ 1638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:27.378620 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:27.380095 (+ 1475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:27.380106 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:27.381469 (+ 1363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:27.381475 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:27.381661 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:27.381697 (+ 36us) client_negotiation.cc:769] Sending connection context
0806 09:29:27.381818 (+ 121us) client_negotiation.cc:240] Negotiation successful
0806 09:29:27.381952 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":815,"mutex_wait_us":4,"thread_start_us":1290,"threads_started":1}
I20260806 09:29:27.382418 387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:27.374649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.3:53177 (local address 127.20.113.1:33839)
0806 09:29:27.374891 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:27.374897 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:27.375605 (+ 708us) server_negotiation.cc:407] Connection header received
0806 09:29:27.375974 (+ 369us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:27.375982 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:27.376075 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:27.376124 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:29:27.377138 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:27.378433 (+ 1295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:27.380256 (+ 1823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:27.381235 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:27.381421 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:27.381499 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:27.381868 (+ 369us) server_negotiation.cc:299] Negotiation successful
0806 09:29:27.382004 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260806 09:29:27.383515 391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:27.375208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.3:39223 (local address 127.20.113.2:45517)
0806 09:29:27.375394 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:27.375400 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:27.376192 (+ 792us) server_negotiation.cc:407] Connection header received
0806 09:29:27.376365 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:27.376370 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:27.376445 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:27.376545 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:29:27.377267 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:27.378441 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:27.380976 (+ 2535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:27.382103 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:27.382272 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:27.382345 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:27.383106 (+ 761us) server_negotiation.cc:299] Negotiation successful
0806 09:29:27.383250 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 09:29:27.383566 434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:27.374601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.2:45517 (local address 127.20.113.3:39223)
0806 09:29:27.376064 (+ 1463us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:27.376081 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:27.376208 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:27.376585 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:27.376590 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:27.376620 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:29:27.377089 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:27.377102 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:27.378603 (+ 1501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:27.378609 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:27.380790 (+ 2181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:27.380806 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:27.382561 (+ 1755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:27.382567 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:27.382918 (+ 351us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:27.382958 (+ 40us) client_negotiation.cc:769] Sending connection context
0806 09:29:27.383053 (+ 95us) client_negotiation.cc:240] Negotiation successful
0806 09:29:27.383186 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1345,"mutex_wait_us":2,"thread_start_us":788,"threads_started":1}
I20260806 09:29:27.383469 32336 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c23e01ea61e43c6b701cab05f3d6a8a" candidate_uuid: "d6930c871eec4d2f9c6efa9c8e742cba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a27e10c55c654384ae155607041dad72" is_pre_election: true
I20260806 09:29:27.383991 32336 raft_consensus.cc:2454] T 2c23e01ea61e43c6b701cab05f3d6a8a P a27e10c55c654384ae155607041dad72 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6930c871eec4d2f9c6efa9c8e742cba in term 0.
I20260806 09:29:27.384405 32548 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c23e01ea61e43c6b701cab05f3d6a8a" candidate_uuid: "d6930c871eec4d2f9c6efa9c8e742cba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9485c4273194777b6ccf02c3e050e6f" is_pre_election: true
I20260806 09:29:27.384761 32548 raft_consensus.cc:2454] T 2c23e01ea61e43c6b701cab05f3d6a8a P d9485c4273194777b6ccf02c3e050e6f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6930c871eec4d2f9c6efa9c8e742cba in term 0.
I20260806 09:29:27.384698 32651 leader_election.cc:302] T 2c23e01ea61e43c6b701cab05f3d6a8a P d6930c871eec4d2f9c6efa9c8e742cba [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a27e10c55c654384ae155607041dad72, d6930c871eec4d2f9c6efa9c8e742cba; no voters:
I20260806 09:29:27.385105 405 raft_consensus.cc:2781] T 2c23e01ea61e43c6b701cab05f3d6a8a P d6930c871eec4d2f9c6efa9c8e742cba [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 09:29:27.385234 405 raft_consensus.cc:482] T 2c23e01ea61e43c6b701cab05f3d6a8a P d6930c871eec4d2f9c6efa9c8e742cba [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 09:29:27.385337 405 raft_consensus.cc:3037] T 2c23e01ea61e43c6b701cab05f3d6a8a P d6930c871eec4d2f9c6efa9c8e742cba [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:29:27.387601 405 raft_consensus.cc:504] T 2c23e01ea61e43c6b701cab05f3d6a8a P d6930c871eec4d2f9c6efa9c8e742cba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a27e10c55c654384ae155607041dad72" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33839 } } peers { permanent_uuid: "d9485c4273194777b6ccf02c3e050e6f" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 45517 } } peers { permanent_uuid: "d6930c871eec4d2f9c6efa9c8e742cba" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 38869 } }
I20260806 09:29:27.388291 405 leader_election.cc:288] T 2c23e01ea61e43c6b701cab05f3d6a8a P d6930c871eec4d2f9c6efa9c8e742cba [CANDIDATE]: Term 1 election: Requested vote from peers a27e10c55c654384ae155607041dad72 (127.20.113.1:33839), d9485c4273194777b6ccf02c3e050e6f (127.20.113.2:45517)
I20260806 09:29:27.389000 32548 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c23e01ea61e43c6b701cab05f3d6a8a" candidate_uuid: "d6930c871eec4d2f9c6efa9c8e742cba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9485c4273194777b6ccf02c3e050e6f"
I20260806 09:29:27.389070 32336 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c23e01ea61e43c6b701cab05f3d6a8a" candidate_uuid: "d6930c871eec4d2f9c6efa9c8e742cba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a27e10c55c654384ae155607041dad72"
I20260806 09:29:27.389220 32548 raft_consensus.cc:3037] T 2c23e01ea61e43c6b701cab05f3d6a8a P d9485c4273194777b6ccf02c3e050e6f [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:29:27.389266 32336 raft_consensus.cc:3037] T 2c23e01ea61e43c6b701cab05f3d6a8a P a27e10c55c654384ae155607041dad72 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:29:27.392325 32548 raft_consensus.cc:2454] T 2c23e01ea61e43c6b701cab05f3d6a8a P d9485c4273194777b6ccf02c3e050e6f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6930c871eec4d2f9c6efa9c8e742cba in term 1.
I20260806 09:29:27.392577 32336 raft_consensus.cc:2454] T 2c23e01ea61e43c6b701cab05f3d6a8a P a27e10c55c654384ae155607041dad72 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6930c871eec4d2f9c6efa9c8e742cba in term 1.
I20260806 09:29:27.392792 32653 leader_election.cc:302] T 2c23e01ea61e43c6b701cab05f3d6a8a P d6930c871eec4d2f9c6efa9c8e742cba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d6930c871eec4d2f9c6efa9c8e742cba, d9485c4273194777b6ccf02c3e050e6f; no voters:
I20260806 09:29:27.393183 405 raft_consensus.cc:2781] T 2c23e01ea61e43c6b701cab05f3d6a8a P d6930c871eec4d2f9c6efa9c8e742cba [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:29:27.393674 405 raft_consensus.cc:686] T 2c23e01ea61e43c6b701cab05f3d6a8a P d6930c871eec4d2f9c6efa9c8e742cba [term 1 LEADER]: Becoming Leader. State: Replica: d6930c871eec4d2f9c6efa9c8e742cba, State: Running, Role: LEADER
I20260806 09:29:27.394202 405 consensus_queue.cc:237] T 2c23e01ea61e43c6b701cab05f3d6a8a P d6930c871eec4d2f9c6efa9c8e742cba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a27e10c55c654384ae155607041dad72" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33839 } } peers { permanent_uuid: "d9485c4273194777b6ccf02c3e050e6f" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 45517 } } peers { permanent_uuid: "d6930c871eec4d2f9c6efa9c8e742cba" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 38869 } }
I20260806 09:29:27.398092 32092 catalog_manager.cc:5495] T 2c23e01ea61e43c6b701cab05f3d6a8a P d6930c871eec4d2f9c6efa9c8e742cba reported cstate change: term changed from 0 to 1, leader changed from <none> to d6930c871eec4d2f9c6efa9c8e742cba (127.20.113.3). New cstate: current_term: 1 leader_uuid: "d6930c871eec4d2f9c6efa9c8e742cba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a27e10c55c654384ae155607041dad72" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33839 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9485c4273194777b6ccf02c3e050e6f" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 45517 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6930c871eec4d2f9c6efa9c8e742cba" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 38869 } health_report { overall_health: HEALTHY } } }
Aug 06 09:29:27 dist-test-slave-45dx krb5kdc[32029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008563, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.113.62@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM
I20260806 09:29:27.462265 445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:27.446787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:45237 (local address 127.0.0.1:53452)
0806 09:29:27.447450 (+ 663us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:27.447471 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:27.447591 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:27.447956 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:27.447959 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:27.448694 (+ 735us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:29:27.448979 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:27.448988 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:27.450575 (+ 1587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:27.450579 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:27.451488 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:27.451499 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:27.452474 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:27.452478 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:27.452597 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:27.453328 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:29:27.453350 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:29:27.455743 (+ 2393us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:29:27.459636 (+ 3893us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:29:27.459646 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:29:27.459651 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:29:27.460144 (+ 493us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:29:27.460581 (+ 437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:29:27.460584 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:29:27.460586 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:29:27.460715 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:29:27.461282 (+ 567us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:29:27.461287 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:29:27.461498 (+ 211us) client_negotiation.cc:769] Sending connection context
0806 09:29:27.461654 (+ 156us) client_negotiation.cc:240] Negotiation successful
0806 09:29:27.461945 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":363,"threads_started":1}
I20260806 09:29:27.462651 32241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:27.446771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53452 (local address 127.20.113.62:45237)
0806 09:29:27.446994 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:27.447001 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:27.447558 (+ 557us) server_negotiation.cc:407] Connection header received
0806 09:29:27.447703 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:27.447709 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:27.447788 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:27.447904 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:27.449137 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:27.450436 (+ 1299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:27.451633 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:27.452279 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:27.452430 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:27.455927 (+ 3497us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:29:27.455933 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:29:27.455938 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:29:27.455974 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:29:27.459508 (+ 3534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:27.460255 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:27.460276 (+ 21us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:27.460279 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:27.460468 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:27.460817 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:27.460822 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:27.460825 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:27.461152 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:29:27.461311 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:27.461818 (+ 507us) server_negotiation.cc:299] Negotiation successful
0806 09:29:27.462118 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":54,"server-negotiator.queue_time_us":76}
I20260806 09:29:27.468150 32092 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.20.113.62@KRBTEST.COM'} at 127.0.0.1:53452:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 09:29:27.469334 32092 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 09:29:27.476567 32308 tablet_service.cc:1467] Processing CreateTablet for tablet 3400141e22354d69b675d40eabddb9d7 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c6a8c78422c44820947af728d4507ebb]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 09:29:27.477326 32308 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3400141e22354d69b675d40eabddb9d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:29:27.477103 32721 tablet_service.cc:1467] Processing CreateTablet for tablet 3400141e22354d69b675d40eabddb9d7 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c6a8c78422c44820947af728d4507ebb]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 09:29:27.477769 32721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3400141e22354d69b675d40eabddb9d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:29:27.480940 32518 tablet_service.cc:1467] Processing CreateTablet for tablet 3400141e22354d69b675d40eabddb9d7 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c6a8c78422c44820947af728d4507ebb]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 09:29:27.481639 32518 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3400141e22354d69b675d40eabddb9d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:29:27.483870 397 tablet_bootstrap.cc:492] T 3400141e22354d69b675d40eabddb9d7 P a27e10c55c654384ae155607041dad72: Bootstrap starting.
I20260806 09:29:27.485555 397 tablet_bootstrap.cc:654] T 3400141e22354d69b675d40eabddb9d7 P a27e10c55c654384ae155607041dad72: Neither blocks nor log segments found. Creating new log.
I20260806 09:29:27.487825 397 tablet_bootstrap.cc:492] T 3400141e22354d69b675d40eabddb9d7 P a27e10c55c654384ae155607041dad72: No bootstrap required, opened a new log
I20260806 09:29:27.488025 397 ts_tablet_manager.cc:1397] T 3400141e22354d69b675d40eabddb9d7 P a27e10c55c654384ae155607041dad72: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260806 09:29:27.488725 397 raft_consensus.cc:348] T 3400141e22354d69b675d40eabddb9d7 P a27e10c55c654384ae155607041dad72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a27e10c55c654384ae155607041dad72" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33839 } } peers { permanent_uuid: "d6930c871eec4d2f9c6efa9c8e742cba" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 38869 } } peers { permanent_uuid: "d9485c4273194777b6ccf02c3e050e6f" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 45517 } }
I20260806 09:29:27.488962 397 raft_consensus.cc:374] T 3400141e22354d69b675d40eabddb9d7 P a27e10c55c654384ae155607041dad72 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:29:27.489075 397 raft_consensus.cc:729] T 3400141e22354d69b675d40eabddb9d7 P a27e10c55c654384ae155607041dad72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a27e10c55c654384ae155607041dad72, State: Initialized, Role: FOLLOWER
I20260806 09:29:27.489327 397 consensus_queue.cc:260] T 3400141e22354d69b675d40eabddb9d7 P a27e10c55c654384ae155607041dad72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a27e10c55c654384ae155607041dad72" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33839 } } peers { permanent_uuid: "d6930c871eec4d2f9c6efa9c8e742cba" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 38869 } } peers { permanent_uuid: "d9485c4273194777b6ccf02c3e050e6f" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 45517 } }
I20260806 09:29:27.489777 403 tablet_replica.cc:439] T 3400141e22354d69b675d40eabddb9d7 P a27e10c55c654384ae155607041dad72: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a27e10c55c654384ae155607041dad72" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33839 } } peers { permanent_uuid: "d6930c871eec4d2f9c6efa9c8e742cba" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 38869 } } peers { permanent_uuid: "d9485c4273194777b6ccf02c3e050e6f" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 45517 } } }
I20260806 09:29:27.490059 403 tablet_replica.cc:442] T 3400141e22354d69b675d40eabddb9d7 P a27e10c55c654384ae155607041dad72: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 09:29:27.490564 397 ts_tablet_manager.cc:1428] T 3400141e22354d69b675d40eabddb9d7 P a27e10c55c654384ae155607041dad72: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 09:29:27.494165 395 tablet_bootstrap.cc:492] T 3400141e22354d69b675d40eabddb9d7 P d6930c871eec4d2f9c6efa9c8e742cba: Bootstrap starting.
I20260806 09:29:27.494287 396 tablet_bootstrap.cc:492] T 3400141e22354d69b675d40eabddb9d7 P d9485c4273194777b6ccf02c3e050e6f: Bootstrap starting.
I20260806 09:29:27.495656 395 tablet_bootstrap.cc:654] T 3400141e22354d69b675d40eabddb9d7 P d6930c871eec4d2f9c6efa9c8e742cba: Neither blocks nor log segments found. Creating new log.
I20260806 09:29:27.496014 396 tablet_bootstrap.cc:654] T 3400141e22354d69b675d40eabddb9d7 P d9485c4273194777b6ccf02c3e050e6f: Neither blocks nor log segments found. Creating new log.
I20260806 09:29:27.497839 395 tablet_bootstrap.cc:492] T 3400141e22354d69b675d40eabddb9d7 P d6930c871eec4d2f9c6efa9c8e742cba: No bootstrap required, opened a new log
I20260806 09:29:27.498034 395 ts_tablet_manager.cc:1397] T 3400141e22354d69b675d40eabddb9d7 P d6930c871eec4d2f9c6efa9c8e742cba: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.002s
I20260806 09:29:27.498615 395 raft_consensus.cc:348] T 3400141e22354d69b675d40eabddb9d7 P d6930c871eec4d2f9c6efa9c8e742cba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a27e10c55c654384ae155607041dad72" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33839 } } peers { permanent_uuid: "d6930c871eec4d2f9c6efa9c8e742cba" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 38869 } } peers { permanent_uuid: "d9485c4273194777b6ccf02c3e050e6f" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 45517 } }
I20260806 09:29:27.498842 395 raft_consensus.cc:374] T 3400141e22354d69b675d40eabddb9d7 P d6930c871eec4d2f9c6efa9c8e742cba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:29:27.498956 395 raft_consensus.cc:729] T 3400141e22354d69b675d40eabddb9d7 P d6930c871eec4d2f9c6efa9c8e742cba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6930c871eec4d2f9c6efa9c8e742cba, State: Initialized, Role: FOLLOWER
I20260806 09:29:27.499203 395 consensus_queue.cc:260] T 3400141e22354d69b675d40eabddb9d7 P d6930c871eec4d2f9c6efa9c8e742cba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a27e10c55c654384ae155607041dad72" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33839 } } peers { permanent_uuid: "d6930c871eec4d2f9c6efa9c8e742cba" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 38869 } } peers { permanent_uuid: "d9485c4273194777b6ccf02c3e050e6f" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 45517 } }
I20260806 09:29:27.499619 405 tablet_replica.cc:439] T 3400141e22354d69b675d40eabddb9d7 P d6930c871eec4d2f9c6efa9c8e742cba: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a27e10c55c654384ae155607041dad72" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33839 } } peers { permanent_uuid: "d6930c871eec4d2f9c6efa9c8e742cba" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 38869 } } peers { permanent_uuid: "d9485c4273194777b6ccf02c3e050e6f" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 45517 } } }
I20260806 09:29:27.499907 405 tablet_replica.cc:442] T 3400141e22354d69b675d40eabddb9d7 P d6930c871eec4d2f9c6efa9c8e742cba: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 09:29:27.500711 395 ts_tablet_manager.cc:1428] T 3400141e22354d69b675d40eabddb9d7 P d6930c871eec4d2f9c6efa9c8e742cba: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
W20260806 09:29:27.505421 32628 tablet.cc:2369] T 2c23e01ea61e43c6b701cab05f3d6a8a P d9485c4273194777b6ccf02c3e050e6f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 09:29:27.505817 396 tablet_bootstrap.cc:492] T 3400141e22354d69b675d40eabddb9d7 P d9485c4273194777b6ccf02c3e050e6f: No bootstrap required, opened a new log
I20260806 09:29:27.505995 396 ts_tablet_manager.cc:1397] T 3400141e22354d69b675d40eabddb9d7 P d9485c4273194777b6ccf02c3e050e6f: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.003s
I20260806 09:29:27.506865 396 raft_consensus.cc:348] T 3400141e22354d69b675d40eabddb9d7 P d9485c4273194777b6ccf02c3e050e6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a27e10c55c654384ae155607041dad72" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33839 } } peers { permanent_uuid: "d6930c871eec4d2f9c6efa9c8e742cba" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 38869 } } peers { permanent_uuid: "d9485c4273194777b6ccf02c3e050e6f" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 45517 } }
I20260806 09:29:27.507138 396 raft_consensus.cc:374] T 3400141e22354d69b675d40eabddb9d7 P d9485c4273194777b6ccf02c3e050e6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:29:27.507241 396 raft_consensus.cc:729] T 3400141e22354d69b675d40eabddb9d7 P d9485c4273194777b6ccf02c3e050e6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9485c4273194777b6ccf02c3e050e6f, State: Initialized, Role: FOLLOWER
I20260806 09:29:27.507524 396 consensus_queue.cc:260] T 3400141e22354d69b675d40eabddb9d7 P d9485c4273194777b6ccf02c3e050e6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a27e10c55c654384ae155607041dad72" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33839 } } peers { permanent_uuid: "d6930c871eec4d2f9c6efa9c8e742cba" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 38869 } } peers { permanent_uuid: "d9485c4273194777b6ccf02c3e050e6f" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 45517 } }
I20260806 09:29:27.508168 396 ts_tablet_manager.cc:1428] T 3400141e22354d69b675d40eabddb9d7 P d9485c4273194777b6ccf02c3e050e6f: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260806 09:29:27.508055 404 tablet_replica.cc:439] T 3400141e22354d69b675d40eabddb9d7 P d9485c4273194777b6ccf02c3e050e6f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a27e10c55c654384ae155607041dad72" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33839 } } peers { permanent_uuid: "d6930c871eec4d2f9c6efa9c8e742cba" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 38869 } } peers { permanent_uuid: "d9485c4273194777b6ccf02c3e050e6f" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 45517 } } }
I20260806 09:29:27.508517 404 tablet_replica.cc:442] T 3400141e22354d69b675d40eabddb9d7 P d9485c4273194777b6ccf02c3e050e6f: This TxnStatusTablet replica's current role is: FOLLOWER
W20260806 09:29:27.538803 353 tablet.cc:2369] T 2c23e01ea61e43c6b701cab05f3d6a8a P d6930c871eec4d2f9c6efa9c8e742cba: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 09:29:27.620213 403 raft_consensus.cc:482] T 3400141e22354d69b675d40eabddb9d7 P a27e10c55c654384ae155607041dad72 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 09:29:27.620487 403 raft_consensus.cc:504] T 3400141e22354d69b675d40eabddb9d7 P a27e10c55c654384ae155607041dad72 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a27e10c55c654384ae155607041dad72" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33839 } } peers { permanent_uuid: "d6930c871eec4d2f9c6efa9c8e742cba" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 38869 } } peers { permanent_uuid: "d9485c4273194777b6ccf02c3e050e6f" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 45517 } }
I20260806 09:29:27.621263 403 leader_election.cc:288] T 3400141e22354d69b675d40eabddb9d7 P a27e10c55c654384ae155607041dad72 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6930c871eec4d2f9c6efa9c8e742cba (127.20.113.3:38869), d9485c4273194777b6ccf02c3e050e6f (127.20.113.2:45517)
I20260806 09:29:27.621871 32548 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3400141e22354d69b675d40eabddb9d7" candidate_uuid: "a27e10c55c654384ae155607041dad72" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9485c4273194777b6ccf02c3e050e6f" is_pre_election: true
I20260806 09:29:27.621918 32748 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3400141e22354d69b675d40eabddb9d7" candidate_uuid: "a27e10c55c654384ae155607041dad72" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6930c871eec4d2f9c6efa9c8e742cba" is_pre_election: true
I20260806 09:29:27.622160 32548 raft_consensus.cc:2454] T 3400141e22354d69b675d40eabddb9d7 P d9485c4273194777b6ccf02c3e050e6f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a27e10c55c654384ae155607041dad72 in term 0.
I20260806 09:29:27.622249 32748 raft_consensus.cc:2454] T 3400141e22354d69b675d40eabddb9d7 P d6930c871eec4d2f9c6efa9c8e742cba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a27e10c55c654384ae155607041dad72 in term 0.
I20260806 09:29:27.622753 32232 leader_election.cc:302] T 3400141e22354d69b675d40eabddb9d7 P a27e10c55c654384ae155607041dad72 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a27e10c55c654384ae155607041dad72, d9485c4273194777b6ccf02c3e050e6f; no voters:
I20260806 09:29:27.623087 403 raft_consensus.cc:2781] T 3400141e22354d69b675d40eabddb9d7 P a27e10c55c654384ae155607041dad72 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 09:29:27.623222 403 raft_consensus.cc:482] T 3400141e22354d69b675d40eabddb9d7 P a27e10c55c654384ae155607041dad72 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 09:29:27.623417 403 raft_consensus.cc:3037] T 3400141e22354d69b675d40eabddb9d7 P a27e10c55c654384ae155607041dad72 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:29:27.626104 403 raft_consensus.cc:504] T 3400141e22354d69b675d40eabddb9d7 P a27e10c55c654384ae155607041dad72 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a27e10c55c654384ae155607041dad72" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33839 } } peers { permanent_uuid: "d6930c871eec4d2f9c6efa9c8e742cba" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 38869 } } peers { permanent_uuid: "d9485c4273194777b6ccf02c3e050e6f" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 45517 } }
I20260806 09:29:27.626748 403 leader_election.cc:288] T 3400141e22354d69b675d40eabddb9d7 P a27e10c55c654384ae155607041dad72 [CANDIDATE]: Term 1 election: Requested vote from peers d6930c871eec4d2f9c6efa9c8e742cba (127.20.113.3:38869), d9485c4273194777b6ccf02c3e050e6f (127.20.113.2:45517)
I20260806 09:29:27.627174 32748 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3400141e22354d69b675d40eabddb9d7" candidate_uuid: "a27e10c55c654384ae155607041dad72" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6930c871eec4d2f9c6efa9c8e742cba"
I20260806 09:29:27.627377 32748 raft_consensus.cc:3037] T 3400141e22354d69b675d40eabddb9d7 P d6930c871eec4d2f9c6efa9c8e742cba [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:29:27.627296 32548 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3400141e22354d69b675d40eabddb9d7" candidate_uuid: "a27e10c55c654384ae155607041dad72" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9485c4273194777b6ccf02c3e050e6f"
I20260806 09:29:27.627548 32548 raft_consensus.cc:3037] T 3400141e22354d69b675d40eabddb9d7 P d9485c4273194777b6ccf02c3e050e6f [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:29:27.630409 32748 raft_consensus.cc:2454] T 3400141e22354d69b675d40eabddb9d7 P d6930c871eec4d2f9c6efa9c8e742cba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a27e10c55c654384ae155607041dad72 in term 1.
I20260806 09:29:27.630398 32548 raft_consensus.cc:2454] T 3400141e22354d69b675d40eabddb9d7 P d9485c4273194777b6ccf02c3e050e6f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a27e10c55c654384ae155607041dad72 in term 1.
I20260806 09:29:27.630817 32232 leader_election.cc:302] T 3400141e22354d69b675d40eabddb9d7 P a27e10c55c654384ae155607041dad72 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a27e10c55c654384ae155607041dad72, d9485c4273194777b6ccf02c3e050e6f; no voters:
I20260806 09:29:27.631197 403 raft_consensus.cc:2781] T 3400141e22354d69b675d40eabddb9d7 P a27e10c55c654384ae155607041dad72 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:29:27.631351 403 raft_consensus.cc:686] T 3400141e22354d69b675d40eabddb9d7 P a27e10c55c654384ae155607041dad72 [term 1 LEADER]: Becoming Leader. State: Replica: a27e10c55c654384ae155607041dad72, State: Running, Role: LEADER
I20260806 09:29:27.631695 403 consensus_queue.cc:237] T 3400141e22354d69b675d40eabddb9d7 P a27e10c55c654384ae155607041dad72 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a27e10c55c654384ae155607041dad72" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33839 } } peers { permanent_uuid: "d6930c871eec4d2f9c6efa9c8e742cba" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 38869 } } peers { permanent_uuid: "d9485c4273194777b6ccf02c3e050e6f" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 45517 } }
I20260806 09:29:27.633612 409 tablet_replica.cc:439] T 3400141e22354d69b675d40eabddb9d7 P a27e10c55c654384ae155607041dad72: TxnStatusTablet state changed. Reason: New leader a27e10c55c654384ae155607041dad72. Latest consensus state: current_term: 1 leader_uuid: "a27e10c55c654384ae155607041dad72" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a27e10c55c654384ae155607041dad72" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33839 } } peers { permanent_uuid: "d6930c871eec4d2f9c6efa9c8e742cba" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 38869 } } peers { permanent_uuid: "d9485c4273194777b6ccf02c3e050e6f" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 45517 } } }
I20260806 09:29:27.633955 409 tablet_replica.cc:442] T 3400141e22354d69b675d40eabddb9d7 P a27e10c55c654384ae155607041dad72: This TxnStatusTablet replica's current role is: LEADER
I20260806 09:29:27.634912 453 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 09:29:27.635164 453 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 09:29:27.635527 32092 catalog_manager.cc:5495] T 3400141e22354d69b675d40eabddb9d7 P a27e10c55c654384ae155607041dad72 reported cstate change: term changed from 0 to 1, leader changed from <none> to a27e10c55c654384ae155607041dad72 (127.20.113.1). New cstate: current_term: 1 leader_uuid: "a27e10c55c654384ae155607041dad72" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a27e10c55c654384ae155607041dad72" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33839 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d6930c871eec4d2f9c6efa9c8e742cba" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 38869 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9485c4273194777b6ccf02c3e050e6f" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 45517 } health_report { overall_health: UNKNOWN } } }
I20260806 09:29:27.653111 387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:27.647964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47232 (local address 127.20.113.1:33839)
0806 09:29:27.648139 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:27.648145 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:27.648187 (+ 42us) server_negotiation.cc:407] Connection header received
0806 09:29:27.648529 (+ 342us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:27.648535 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:27.648617 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:27.648715 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 09:29:27.649094 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:27.649836 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:27.651112 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:27.651637 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:27.651792 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:27.652244 (+ 452us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 09:29:27.652444 (+ 200us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 09:29:27.652546 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:27.652689 (+ 143us) server_negotiation.cc:299] Negotiation successful
0806 09:29:27.652778 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 09:29:27.653120 445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:27.647848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.1:33839 (local address 127.0.0.1:47232)
0806 09:29:27.648110 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:27.648134 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:27.648288 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:27.648756 (+ 468us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:27.648763 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:27.648776 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 09:29:27.648957 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:27.648965 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:27.649986 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:27.649993 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:27.650969 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:27.650981 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:27.651814 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:27.651820 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:27.651999 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:27.652055 (+ 56us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 09:29:27.652547 (+ 492us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 09:29:27.652571 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 09:29:27.652654 (+ 83us) client_negotiation.cc:240] Negotiation successful
0806 09:29:27.652817 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73}
I20260806 09:29:27.657160 32748 raft_consensus.cc:1261] T 3400141e22354d69b675d40eabddb9d7 P d6930c871eec4d2f9c6efa9c8e742cba [term 1 FOLLOWER]: Refusing update from remote peer a27e10c55c654384ae155607041dad72: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 09:29:27.657202 32548 raft_consensus.cc:1261] T 3400141e22354d69b675d40eabddb9d7 P d9485c4273194777b6ccf02c3e050e6f [term 1 FOLLOWER]: Refusing update from remote peer a27e10c55c654384ae155607041dad72: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 09:29:27.658007 409 consensus_queue.cc:1035] T 3400141e22354d69b675d40eabddb9d7 P a27e10c55c654384ae155607041dad72 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6930c871eec4d2f9c6efa9c8e742cba" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 38869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 09:29:27.658711 403 consensus_queue.cc:1035] T 3400141e22354d69b675d40eabddb9d7 P a27e10c55c654384ae155607041dad72 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9485c4273194777b6ccf02c3e050e6f" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 45517 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 09:29:27.662671 404 tablet_replica.cc:439] T 3400141e22354d69b675d40eabddb9d7 P d9485c4273194777b6ccf02c3e050e6f: TxnStatusTablet state changed. Reason: New leader a27e10c55c654384ae155607041dad72. Latest consensus state: current_term: 1 leader_uuid: "a27e10c55c654384ae155607041dad72" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a27e10c55c654384ae155607041dad72" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33839 } } peers { permanent_uuid: "d6930c871eec4d2f9c6efa9c8e742cba" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 38869 } } peers { permanent_uuid: "d9485c4273194777b6ccf02c3e050e6f" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 45517 } } }
I20260806 09:29:27.662974 404 tablet_replica.cc:442] T 3400141e22354d69b675d40eabddb9d7 P d9485c4273194777b6ccf02c3e050e6f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 09:29:27.664978 409 tablet_replica.cc:439] T 3400141e22354d69b675d40eabddb9d7 P a27e10c55c654384ae155607041dad72: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a27e10c55c654384ae155607041dad72" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a27e10c55c654384ae155607041dad72" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33839 } } peers { permanent_uuid: "d6930c871eec4d2f9c6efa9c8e742cba" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 38869 } } peers { permanent_uuid: "d9485c4273194777b6ccf02c3e050e6f" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 45517 } } }
I20260806 09:29:27.665215 409 tablet_replica.cc:442] T 3400141e22354d69b675d40eabddb9d7 P a27e10c55c654384ae155607041dad72: This TxnStatusTablet replica's current role is: LEADER
I20260806 09:29:27.666188 417 mvcc.cc:205] Tried to move back new op lower bound from 7315491093117067264 to 7315491093022318592. Current Snapshot: MvccSnapshot[applied={T|T < 7315491093117067264 or (T in {7315491093117067264})}]
I20260806 09:29:27.666886 404 tablet_replica.cc:439] T 3400141e22354d69b675d40eabddb9d7 P d9485c4273194777b6ccf02c3e050e6f: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a27e10c55c654384ae155607041dad72" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a27e10c55c654384ae155607041dad72" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33839 } } peers { permanent_uuid: "d6930c871eec4d2f9c6efa9c8e742cba" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 38869 } } peers { permanent_uuid: "d9485c4273194777b6ccf02c3e050e6f" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 45517 } } }
I20260806 09:29:27.667114 404 tablet_replica.cc:442] T 3400141e22354d69b675d40eabddb9d7 P d9485c4273194777b6ccf02c3e050e6f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 09:29:27.671835 405 tablet_replica.cc:439] T 3400141e22354d69b675d40eabddb9d7 P d6930c871eec4d2f9c6efa9c8e742cba: TxnStatusTablet state changed. Reason: New leader a27e10c55c654384ae155607041dad72. Latest consensus state: current_term: 1 leader_uuid: "a27e10c55c654384ae155607041dad72" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a27e10c55c654384ae155607041dad72" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33839 } } peers { permanent_uuid: "d6930c871eec4d2f9c6efa9c8e742cba" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 38869 } } peers { permanent_uuid: "d9485c4273194777b6ccf02c3e050e6f" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 45517 } } }
I20260806 09:29:27.672111 405 tablet_replica.cc:442] T 3400141e22354d69b675d40eabddb9d7 P d6930c871eec4d2f9c6efa9c8e742cba: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 09:29:27.673005 409 tablet_replica.cc:439] T 3400141e22354d69b675d40eabddb9d7 P a27e10c55c654384ae155607041dad72: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a27e10c55c654384ae155607041dad72" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a27e10c55c654384ae155607041dad72" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33839 } } peers { permanent_uuid: "d6930c871eec4d2f9c6efa9c8e742cba" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 38869 } } peers { permanent_uuid: "d9485c4273194777b6ccf02c3e050e6f" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 45517 } } }
I20260806 09:29:27.673238 409 tablet_replica.cc:442] T 3400141e22354d69b675d40eabddb9d7 P a27e10c55c654384ae155607041dad72: This TxnStatusTablet replica's current role is: LEADER
I20260806 09:29:27.674811 405 tablet_replica.cc:439] T 3400141e22354d69b675d40eabddb9d7 P d6930c871eec4d2f9c6efa9c8e742cba: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a27e10c55c654384ae155607041dad72" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a27e10c55c654384ae155607041dad72" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33839 } } peers { permanent_uuid: "d6930c871eec4d2f9c6efa9c8e742cba" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 38869 } } peers { permanent_uuid: "d9485c4273194777b6ccf02c3e050e6f" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 45517 } } }
I20260806 09:29:27.675132 405 tablet_replica.cc:442] T 3400141e22354d69b675d40eabddb9d7 P d6930c871eec4d2f9c6efa9c8e742cba: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 09:29:27.683012 461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:27.674034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50664 (local address 127.20.113.3:38869)
0806 09:29:27.676209 (+ 2175us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:27.676215 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:27.676234 (+ 19us) server_negotiation.cc:407] Connection header received
0806 09:29:27.676380 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:27.676386 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:27.676502 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:27.677297 (+ 795us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 09:29:27.677533 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:27.678671 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:27.680667 (+ 1996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:27.681080 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:27.681207 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:27.681778 (+ 571us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 09:29:27.682321 (+ 543us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 09:29:27.682441 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:27.682582 (+ 141us) server_negotiation.cc:299] Negotiation successful
0806 09:29:27.682645 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2051,"thread_start_us":1885,"threads_started":1}
I20260806 09:29:27.698998 387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:27.693027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.3:54285 (local address 127.20.113.1:33839)
0806 09:29:27.693220 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:27.693228 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:27.693736 (+ 508us) server_negotiation.cc:407] Connection header received
0806 09:29:27.694034 (+ 298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:27.694039 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:27.694103 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:27.694197 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 09:29:27.694686 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:27.695462 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:27.696872 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:27.697577 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:27.697692 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:27.698193 (+ 501us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 09:29:27.698356 (+ 163us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 09:29:27.698444 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:27.698595 (+ 151us) server_negotiation.cc:299] Negotiation successful
0806 09:29:27.698671 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 09:29:27.699530 466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:27.692994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.1:33839 (local address 127.20.113.3:54285)
0806 09:29:27.693627 (+ 633us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:27.693652 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:27.693878 (+ 226us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:27.694214 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:27.694218 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:27.694229 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 09:29:27.694535 (+ 306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:27.694543 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:27.695604 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:27.695610 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:27.696736 (+ 1126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:27.696748 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:27.697773 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:27.697780 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:27.697956 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:27.698020 (+ 64us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 09:29:27.698467 (+ 447us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 09:29:27.698480 (+ 13us) client_negotiation.cc:769] Sending connection context
0806 09:29:27.698559 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 09:29:27.698690 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":343,"threads_started":1}
I20260806 09:29:27.710956 32548 raft_consensus.cc:1261] T 2c23e01ea61e43c6b701cab05f3d6a8a P d9485c4273194777b6ccf02c3e050e6f [term 1 FOLLOWER]: Refusing update from remote peer d6930c871eec4d2f9c6efa9c8e742cba: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 09:29:27.711051 32336 raft_consensus.cc:1261] T 2c23e01ea61e43c6b701cab05f3d6a8a P a27e10c55c654384ae155607041dad72 [term 1 FOLLOWER]: Refusing update from remote peer d6930c871eec4d2f9c6efa9c8e742cba: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 09:29:27.711628 437 consensus_queue.cc:1035] T 2c23e01ea61e43c6b701cab05f3d6a8a P d6930c871eec4d2f9c6efa9c8e742cba [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9485c4273194777b6ccf02c3e050e6f" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 45517 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 09:29:27.712277 405 consensus_queue.cc:1035] T 2c23e01ea61e43c6b701cab05f3d6a8a P d6930c871eec4d2f9c6efa9c8e742cba [LEADER]: Connected to new peer: Peer: permanent_uuid: "a27e10c55c654384ae155607041dad72" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 09:29:27.725713 416 mvcc.cc:205] Tried to move back new op lower bound from 7315491093337669632 to 7315491092051464192. Current Snapshot: MvccSnapshot[applied={T|T < 7315491093337669632}]
I20260806 09:29:27.765731 461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:27.758013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.1:36287 (local address 127.20.113.3:38869)
0806 09:29:27.758213 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:27.758219 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:27.760089 (+ 1870us) server_negotiation.cc:407] Connection header received
0806 09:29:27.760380 (+ 291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:27.760395 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:27.760484 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:27.760573 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 09:29:27.761101 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:27.761967 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:27.763698 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:27.764271 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:27.764434 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:27.765077 (+ 643us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 09:29:27.765195 (+ 118us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 09:29:27.765267 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:27.765440 (+ 173us) server_negotiation.cc:299] Negotiation successful
0806 09:29:27.765503 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260806 09:29:27.766233 476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:27.758128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.3:38869 (local address 127.20.113.1:36287)
0806 09:29:27.759987 (+ 1859us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:27.760007 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:27.760205 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:27.760602 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:27.760606 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:27.760618 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 09:29:27.760945 (+ 327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:27.760955 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:27.762106 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:27.762112 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:27.763559 (+ 1447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:27.763572 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:27.764491 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:27.764499 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:27.764835 (+ 336us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:27.764908 (+ 73us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 09:29:27.765299 (+ 391us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 09:29:27.765325 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 09:29:27.765411 (+ 86us) client_negotiation.cc:240] Negotiation successful
0806 09:29:27.765558 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1603,"thread_start_us":1399,"threads_started":1}
I20260806 09:29:27.824601 32092 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53436:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 09:29:27.824877 32092 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53436:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 09:29:27.828715 32092 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 81679e1de52c40d883ea9f9173070aa7: Sending DeleteTablet for 3 replicas of tablet 2c23e01ea61e43c6b701cab05f3d6a8a
I20260806 09:29:27.829521 32308 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c23e01ea61e43c6b701cab05f3d6a8a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:29:27 UTC) from {username='oryx', principal='oryx/127.20.113.62@KRBTEST.COM'} at 127.0.0.1:47208
I20260806 09:29:27.829818 32518 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c23e01ea61e43c6b701cab05f3d6a8a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:29:27 UTC) from {username='oryx', principal='oryx/127.20.113.62@KRBTEST.COM'} at 127.0.0.1:52408
I20260806 09:29:27.830698 32721 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c23e01ea61e43c6b701cab05f3d6a8a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:29:27 UTC) from {username='oryx', principal='oryx/127.20.113.62@KRBTEST.COM'} at 127.0.0.1:50648
I20260806 09:29:27.855027 482 tablet_replica.cc:330] T 2c23e01ea61e43c6b701cab05f3d6a8a P d6930c871eec4d2f9c6efa9c8e742cba: stopping tablet replica
I20260806 09:29:27.855558 481 tablet_replica.cc:330] T 2c23e01ea61e43c6b701cab05f3d6a8a P d9485c4273194777b6ccf02c3e050e6f: stopping tablet replica
I20260806 09:29:27.855870 482 raft_consensus.cc:2229] T 2c23e01ea61e43c6b701cab05f3d6a8a P d6930c871eec4d2f9c6efa9c8e742cba [term 1 LEADER]: Raft consensus shutting down.
I20260806 09:29:27.856599 482 raft_consensus.cc:2258] T 2c23e01ea61e43c6b701cab05f3d6a8a P d6930c871eec4d2f9c6efa9c8e742cba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 09:29:27.856915 481 raft_consensus.cc:2229] T 2c23e01ea61e43c6b701cab05f3d6a8a P d9485c4273194777b6ccf02c3e050e6f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 09:29:27.857250 481 raft_consensus.cc:2258] T 2c23e01ea61e43c6b701cab05f3d6a8a P d9485c4273194777b6ccf02c3e050e6f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 09:29:27.859200 481 ts_tablet_manager.cc:1905] T 2c23e01ea61e43c6b701cab05f3d6a8a P d9485c4273194777b6ccf02c3e050e6f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 09:29:27.859758 482 ts_tablet_manager.cc:1905] T 2c23e01ea61e43c6b701cab05f3d6a8a P d6930c871eec4d2f9c6efa9c8e742cba: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 09:29:27.876433 482 ts_tablet_manager.cc:1918] T 2c23e01ea61e43c6b701cab05f3d6a8a P d6930c871eec4d2f9c6efa9c8e742cba: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 09:29:27.877213 480 tablet_replica.cc:330] T 2c23e01ea61e43c6b701cab05f3d6a8a P a27e10c55c654384ae155607041dad72: stopping tablet replica
I20260806 09:29:27.877537 480 raft_consensus.cc:2229] T 2c23e01ea61e43c6b701cab05f3d6a8a P a27e10c55c654384ae155607041dad72 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 09:29:27.878067 480 raft_consensus.cc:2258] T 2c23e01ea61e43c6b701cab05f3d6a8a P a27e10c55c654384ae155607041dad72 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 09:29:27.879539 480 ts_tablet_manager.cc:1905] T 2c23e01ea61e43c6b701cab05f3d6a8a P a27e10c55c654384ae155607041dad72: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 09:29:27.888007 480 ts_tablet_manager.cc:1918] T 2c23e01ea61e43c6b701cab05f3d6a8a P a27e10c55c654384ae155607041dad72: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 09:29:27.888510 481 ts_tablet_manager.cc:1918] T 2c23e01ea61e43c6b701cab05f3d6a8a P d9485c4273194777b6ccf02c3e050e6f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 09:29:27.888732 481 log.cc:1196] T 2c23e01ea61e43c6b701cab05f3d6a8a P d9485c4273194777b6ccf02c3e050e6f: Deleting WAL directory at /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786008495763054-20932-0/minicluster-data/ts-1/wal/wals/2c23e01ea61e43c6b701cab05f3d6a8a
I20260806 09:29:27.889397 481 ts_tablet_manager.cc:1939] T 2c23e01ea61e43c6b701cab05f3d6a8a P d9485c4273194777b6ccf02c3e050e6f: Deleting consensus metadata
I20260806 09:29:27.889622 482 log.cc:1196] T 2c23e01ea61e43c6b701cab05f3d6a8a P d6930c871eec4d2f9c6efa9c8e742cba: Deleting WAL directory at /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786008495763054-20932-0/minicluster-data/ts-2/wal/wals/2c23e01ea61e43c6b701cab05f3d6a8a
I20260806 09:29:27.890152 482 ts_tablet_manager.cc:1939] T 2c23e01ea61e43c6b701cab05f3d6a8a P d6930c871eec4d2f9c6efa9c8e742cba: Deleting consensus metadata
I20260806 09:29:27.891520 32077 catalog_manager.cc:4841] TS d9485c4273194777b6ccf02c3e050e6f (127.20.113.2:45517): tablet 2c23e01ea61e43c6b701cab05f3d6a8a (table test-table [id=60af8cd5737540b0afbbadc4b847c9cf]) successfully deleted
I20260806 09:29:27.893092 32075 catalog_manager.cc:4841] TS d6930c871eec4d2f9c6efa9c8e742cba (127.20.113.3:38869): tablet 2c23e01ea61e43c6b701cab05f3d6a8a (table test-table [id=60af8cd5737540b0afbbadc4b847c9cf]) successfully deleted
I20260806 09:29:27.894045 480 log.cc:1196] T 2c23e01ea61e43c6b701cab05f3d6a8a P a27e10c55c654384ae155607041dad72: Deleting WAL directory at /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786008495763054-20932-0/minicluster-data/ts-0/wal/wals/2c23e01ea61e43c6b701cab05f3d6a8a
I20260806 09:29:27.894687 480 ts_tablet_manager.cc:1939] T 2c23e01ea61e43c6b701cab05f3d6a8a P a27e10c55c654384ae155607041dad72: Deleting consensus metadata
I20260806 09:29:27.895898 32075 catalog_manager.cc:4841] TS a27e10c55c654384ae155607041dad72 (127.20.113.1:33839): tablet 2c23e01ea61e43c6b701cab05f3d6a8a (table test-table [id=60af8cd5737540b0afbbadc4b847c9cf]) successfully deleted
Aug 06 09:29:27 dist-test-slave-45dx krb5kdc[32029](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008567, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 09:29:27 dist-test-slave-45dx krb5kdc[32029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.20.113.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 09:29:27 dist-test-slave-45dx krb5kdc[32029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.20.113.62@KRBTEST.COM, Server not found in Kerberos database
W20260806 09:29:27.945468 32241 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 09:29:27.934256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53468 (local address 127.20.113.62:45237)
0806 09:29:27.934507 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:27.934517 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:27.935063 (+ 546us) server_negotiation.cc:407] Connection header received
0806 09:29:27.935306 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:27.935310 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:27.935380 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:27.935494 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:27.937005 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:27.937735 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:27.938997 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:27.939462 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:27.939598 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:27.945192 (+ 5594us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53468: BlockingRecv error: recv got EOF from 127.0.0.1:53468 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 06 09:29:27 dist-test-slave-45dx krb5kdc[32029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008567, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM
I20260806 09:29:27.963860 32241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:27.948862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53472 (local address 127.20.113.62:45237)
0806 09:29:27.949056 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:27.949062 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:27.950421 (+ 1359us) server_negotiation.cc:407] Connection header received
0806 09:29:27.950601 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:27.950607 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:27.950679 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:27.950779 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:27.951860 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:27.952658 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:27.953802 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:27.954215 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:27.954351 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:27.958512 (+ 4161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:29:27.958517 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:29:27.958520 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:29:27.958553 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:29:27.961270 (+ 2717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:27.961901 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:27.961907 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:27.961912 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:27.962039 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:27.962417 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:27.962422 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:27.962425 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:27.962684 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:29:27.962933 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:27.963231 (+ 298us) server_negotiation.cc:299] Negotiation successful
0806 09:29:27.963480 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 09:29:27.967463 32092 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53472:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 09:29:27.968179 32092 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 09:29:27.974792 32308 tablet_service.cc:1467] Processing CreateTablet for tablet 9eb5c43e40ef497ea05097c6ad368d22 (DEFAULT_TABLE table=test-table [id=79343b28556c49ee83207f48190ea096]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:29:27.975287 32308 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9eb5c43e40ef497ea05097c6ad368d22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:29:27.975184 32518 tablet_service.cc:1467] Processing CreateTablet for tablet 9eb5c43e40ef497ea05097c6ad368d22 (DEFAULT_TABLE table=test-table [id=79343b28556c49ee83207f48190ea096]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:29:27.975613 32518 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9eb5c43e40ef497ea05097c6ad368d22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:29:27.981139 32721 tablet_service.cc:1467] Processing CreateTablet for tablet 9eb5c43e40ef497ea05097c6ad368d22 (DEFAULT_TABLE table=test-table [id=79343b28556c49ee83207f48190ea096]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:29:27.981580 32721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9eb5c43e40ef497ea05097c6ad368d22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:29:27.984566 397 tablet_bootstrap.cc:492] T 9eb5c43e40ef497ea05097c6ad368d22 P a27e10c55c654384ae155607041dad72: Bootstrap starting.
I20260806 09:29:27.985878 397 tablet_bootstrap.cc:654] T 9eb5c43e40ef497ea05097c6ad368d22 P a27e10c55c654384ae155607041dad72: Neither blocks nor log segments found. Creating new log.
I20260806 09:29:27.989655 395 tablet_bootstrap.cc:492] T 9eb5c43e40ef497ea05097c6ad368d22 P d6930c871eec4d2f9c6efa9c8e742cba: Bootstrap starting.
I20260806 09:29:27.991381 395 tablet_bootstrap.cc:654] T 9eb5c43e40ef497ea05097c6ad368d22 P d6930c871eec4d2f9c6efa9c8e742cba: Neither blocks nor log segments found. Creating new log.
I20260806 09:29:27.992678 396 tablet_bootstrap.cc:492] T 9eb5c43e40ef497ea05097c6ad368d22 P d9485c4273194777b6ccf02c3e050e6f: Bootstrap starting.
I20260806 09:29:27.994261 396 tablet_bootstrap.cc:654] T 9eb5c43e40ef497ea05097c6ad368d22 P d9485c4273194777b6ccf02c3e050e6f: Neither blocks nor log segments found. Creating new log.
I20260806 09:29:27.994984 397 tablet_bootstrap.cc:492] T 9eb5c43e40ef497ea05097c6ad368d22 P a27e10c55c654384ae155607041dad72: No bootstrap required, opened a new log
I20260806 09:29:27.995163 397 ts_tablet_manager.cc:1397] T 9eb5c43e40ef497ea05097c6ad368d22 P a27e10c55c654384ae155607041dad72: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.001s
I20260806 09:29:27.996276 397 raft_consensus.cc:348] T 9eb5c43e40ef497ea05097c6ad368d22 P a27e10c55c654384ae155607041dad72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a27e10c55c654384ae155607041dad72" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33839 } } peers { permanent_uuid: "d9485c4273194777b6ccf02c3e050e6f" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 45517 } } peers { permanent_uuid: "d6930c871eec4d2f9c6efa9c8e742cba" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 38869 } }
I20260806 09:29:27.997048 397 raft_consensus.cc:374] T 9eb5c43e40ef497ea05097c6ad368d22 P a27e10c55c654384ae155607041dad72 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:29:27.997257 397 raft_consensus.cc:729] T 9eb5c43e40ef497ea05097c6ad368d22 P a27e10c55c654384ae155607041dad72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a27e10c55c654384ae155607041dad72, State: Initialized, Role: FOLLOWER
I20260806 09:29:27.997661 397 consensus_queue.cc:260] T 9eb5c43e40ef497ea05097c6ad368d22 P a27e10c55c654384ae155607041dad72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a27e10c55c654384ae155607041dad72" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33839 } } peers { permanent_uuid: "d9485c4273194777b6ccf02c3e050e6f" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 45517 } } peers { permanent_uuid: "d6930c871eec4d2f9c6efa9c8e742cba" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 38869 } }
I20260806 09:29:28.001639 397 ts_tablet_manager.cc:1428] T 9eb5c43e40ef497ea05097c6ad368d22 P a27e10c55c654384ae155607041dad72: Time spent starting tablet: real 0.006s user 0.000s sys 0.003s
I20260806 09:29:28.007536 395 tablet_bootstrap.cc:492] T 9eb5c43e40ef497ea05097c6ad368d22 P d6930c871eec4d2f9c6efa9c8e742cba: No bootstrap required, opened a new log
I20260806 09:29:28.007727 395 ts_tablet_manager.cc:1397] T 9eb5c43e40ef497ea05097c6ad368d22 P d6930c871eec4d2f9c6efa9c8e742cba: Time spent bootstrapping tablet: real 0.018s user 0.012s sys 0.004s
I20260806 09:29:28.008678 395 raft_consensus.cc:348] T 9eb5c43e40ef497ea05097c6ad368d22 P d6930c871eec4d2f9c6efa9c8e742cba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a27e10c55c654384ae155607041dad72" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33839 } } peers { permanent_uuid: "d9485c4273194777b6ccf02c3e050e6f" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 45517 } } peers { permanent_uuid: "d6930c871eec4d2f9c6efa9c8e742cba" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 38869 } }
I20260806 09:29:28.009024 395 raft_consensus.cc:374] T 9eb5c43e40ef497ea05097c6ad368d22 P d6930c871eec4d2f9c6efa9c8e742cba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:29:28.009159 395 raft_consensus.cc:729] T 9eb5c43e40ef497ea05097c6ad368d22 P d6930c871eec4d2f9c6efa9c8e742cba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6930c871eec4d2f9c6efa9c8e742cba, State: Initialized, Role: FOLLOWER
I20260806 09:29:28.009435 395 consensus_queue.cc:260] T 9eb5c43e40ef497ea05097c6ad368d22 P d6930c871eec4d2f9c6efa9c8e742cba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a27e10c55c654384ae155607041dad72" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33839 } } peers { permanent_uuid: "d9485c4273194777b6ccf02c3e050e6f" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 45517 } } peers { permanent_uuid: "d6930c871eec4d2f9c6efa9c8e742cba" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 38869 } }
I20260806 09:29:28.011370 395 ts_tablet_manager.cc:1428] T 9eb5c43e40ef497ea05097c6ad368d22 P d6930c871eec4d2f9c6efa9c8e742cba: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260806 09:29:28.015679 396 tablet_bootstrap.cc:492] T 9eb5c43e40ef497ea05097c6ad368d22 P d9485c4273194777b6ccf02c3e050e6f: No bootstrap required, opened a new log
I20260806 09:29:28.015869 396 ts_tablet_manager.cc:1397] T 9eb5c43e40ef497ea05097c6ad368d22 P d9485c4273194777b6ccf02c3e050e6f: Time spent bootstrapping tablet: real 0.023s user 0.000s sys 0.010s
I20260806 09:29:28.016705 396 raft_consensus.cc:348] T 9eb5c43e40ef497ea05097c6ad368d22 P d9485c4273194777b6ccf02c3e050e6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a27e10c55c654384ae155607041dad72" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33839 } } peers { permanent_uuid: "d9485c4273194777b6ccf02c3e050e6f" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 45517 } } peers { permanent_uuid: "d6930c871eec4d2f9c6efa9c8e742cba" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 38869 } }
I20260806 09:29:28.016944 396 raft_consensus.cc:374] T 9eb5c43e40ef497ea05097c6ad368d22 P d9485c4273194777b6ccf02c3e050e6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:29:28.017035 396 raft_consensus.cc:729] T 9eb5c43e40ef497ea05097c6ad368d22 P d9485c4273194777b6ccf02c3e050e6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9485c4273194777b6ccf02c3e050e6f, State: Initialized, Role: FOLLOWER
I20260806 09:29:28.017290 396 consensus_queue.cc:260] T 9eb5c43e40ef497ea05097c6ad368d22 P d9485c4273194777b6ccf02c3e050e6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a27e10c55c654384ae155607041dad72" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33839 } } peers { permanent_uuid: "d9485c4273194777b6ccf02c3e050e6f" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 45517 } } peers { permanent_uuid: "d6930c871eec4d2f9c6efa9c8e742cba" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 38869 } }
I20260806 09:29:28.019574 396 ts_tablet_manager.cc:1428] T 9eb5c43e40ef497ea05097c6ad368d22 P d9485c4273194777b6ccf02c3e050e6f: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260806 09:29:28.152482 409 raft_consensus.cc:482] T 9eb5c43e40ef497ea05097c6ad368d22 P a27e10c55c654384ae155607041dad72 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 09:29:28.152716 409 raft_consensus.cc:504] T 9eb5c43e40ef497ea05097c6ad368d22 P a27e10c55c654384ae155607041dad72 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a27e10c55c654384ae155607041dad72" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33839 } } peers { permanent_uuid: "d9485c4273194777b6ccf02c3e050e6f" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 45517 } } peers { permanent_uuid: "d6930c871eec4d2f9c6efa9c8e742cba" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 38869 } }
I20260806 09:29:28.153570 409 leader_election.cc:288] T 9eb5c43e40ef497ea05097c6ad368d22 P a27e10c55c654384ae155607041dad72 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d9485c4273194777b6ccf02c3e050e6f (127.20.113.2:45517), d6930c871eec4d2f9c6efa9c8e742cba (127.20.113.3:38869)
I20260806 09:29:28.154383 32551 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9eb5c43e40ef497ea05097c6ad368d22" candidate_uuid: "a27e10c55c654384ae155607041dad72" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9485c4273194777b6ccf02c3e050e6f" is_pre_election: true
I20260806 09:29:28.154496 32748 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9eb5c43e40ef497ea05097c6ad368d22" candidate_uuid: "a27e10c55c654384ae155607041dad72" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6930c871eec4d2f9c6efa9c8e742cba" is_pre_election: true
I20260806 09:29:28.154670 32551 raft_consensus.cc:2454] T 9eb5c43e40ef497ea05097c6ad368d22 P d9485c4273194777b6ccf02c3e050e6f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a27e10c55c654384ae155607041dad72 in term 0.
I20260806 09:29:28.154757 32748 raft_consensus.cc:2454] T 9eb5c43e40ef497ea05097c6ad368d22 P d6930c871eec4d2f9c6efa9c8e742cba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a27e10c55c654384ae155607041dad72 in term 0.
I20260806 09:29:28.155205 32232 leader_election.cc:302] T 9eb5c43e40ef497ea05097c6ad368d22 P a27e10c55c654384ae155607041dad72 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a27e10c55c654384ae155607041dad72, d9485c4273194777b6ccf02c3e050e6f; no voters:
I20260806 09:29:28.155622 409 raft_consensus.cc:2781] T 9eb5c43e40ef497ea05097c6ad368d22 P a27e10c55c654384ae155607041dad72 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 09:29:28.155758 409 raft_consensus.cc:482] T 9eb5c43e40ef497ea05097c6ad368d22 P a27e10c55c654384ae155607041dad72 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 09:29:28.155860 409 raft_consensus.cc:3037] T 9eb5c43e40ef497ea05097c6ad368d22 P a27e10c55c654384ae155607041dad72 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:29:28.159201 409 raft_consensus.cc:504] T 9eb5c43e40ef497ea05097c6ad368d22 P a27e10c55c654384ae155607041dad72 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a27e10c55c654384ae155607041dad72" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33839 } } peers { permanent_uuid: "d9485c4273194777b6ccf02c3e050e6f" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 45517 } } peers { permanent_uuid: "d6930c871eec4d2f9c6efa9c8e742cba" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 38869 } }
I20260806 09:29:28.160468 32748 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9eb5c43e40ef497ea05097c6ad368d22" candidate_uuid: "a27e10c55c654384ae155607041dad72" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6930c871eec4d2f9c6efa9c8e742cba"
I20260806 09:29:28.160487 32551 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9eb5c43e40ef497ea05097c6ad368d22" candidate_uuid: "a27e10c55c654384ae155607041dad72" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9485c4273194777b6ccf02c3e050e6f"
I20260806 09:29:28.160692 32748 raft_consensus.cc:3037] T 9eb5c43e40ef497ea05097c6ad368d22 P d6930c871eec4d2f9c6efa9c8e742cba [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:29:28.160743 32551 raft_consensus.cc:3037] T 9eb5c43e40ef497ea05097c6ad368d22 P d9485c4273194777b6ccf02c3e050e6f [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:29:28.164026 32748 raft_consensus.cc:2454] T 9eb5c43e40ef497ea05097c6ad368d22 P d6930c871eec4d2f9c6efa9c8e742cba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a27e10c55c654384ae155607041dad72 in term 1.
I20260806 09:29:28.164435 32551 raft_consensus.cc:2454] T 9eb5c43e40ef497ea05097c6ad368d22 P d9485c4273194777b6ccf02c3e050e6f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a27e10c55c654384ae155607041dad72 in term 1.
I20260806 09:29:28.164597 32230 leader_election.cc:302] T 9eb5c43e40ef497ea05097c6ad368d22 P a27e10c55c654384ae155607041dad72 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a27e10c55c654384ae155607041dad72, d6930c871eec4d2f9c6efa9c8e742cba; no voters:
I20260806 09:29:28.165004 403 raft_consensus.cc:2781] T 9eb5c43e40ef497ea05097c6ad368d22 P a27e10c55c654384ae155607041dad72 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:29:28.165156 403 raft_consensus.cc:686] T 9eb5c43e40ef497ea05097c6ad368d22 P a27e10c55c654384ae155607041dad72 [term 1 LEADER]: Becoming Leader. State: Replica: a27e10c55c654384ae155607041dad72, State: Running, Role: LEADER
I20260806 09:29:28.165495 403 consensus_queue.cc:237] T 9eb5c43e40ef497ea05097c6ad368d22 P a27e10c55c654384ae155607041dad72 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a27e10c55c654384ae155607041dad72" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33839 } } peers { permanent_uuid: "d9485c4273194777b6ccf02c3e050e6f" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 45517 } } peers { permanent_uuid: "d6930c871eec4d2f9c6efa9c8e742cba" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 38869 } }
I20260806 09:29:28.168118 409 leader_election.cc:288] T 9eb5c43e40ef497ea05097c6ad368d22 P a27e10c55c654384ae155607041dad72 [CANDIDATE]: Term 1 election: Requested vote from peers d9485c4273194777b6ccf02c3e050e6f (127.20.113.2:45517), d6930c871eec4d2f9c6efa9c8e742cba (127.20.113.3:38869)
I20260806 09:29:28.175937 32092 catalog_manager.cc:5495] T 9eb5c43e40ef497ea05097c6ad368d22 P a27e10c55c654384ae155607041dad72 reported cstate change: term changed from 0 to 1, leader changed from <none> to a27e10c55c654384ae155607041dad72 (127.20.113.1). New cstate: current_term: 1 leader_uuid: "a27e10c55c654384ae155607041dad72" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a27e10c55c654384ae155607041dad72" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33839 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9485c4273194777b6ccf02c3e050e6f" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 45517 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6930c871eec4d2f9c6efa9c8e742cba" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 38869 } health_report { overall_health: UNKNOWN } } }
I20260806 09:29:28.219841 509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:28.212218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47234 (local address 127.20.113.1:33839)
0806 09:29:28.213202 (+ 984us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:28.213210 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:28.213233 (+ 23us) server_negotiation.cc:407] Connection header received
0806 09:29:28.213300 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:28.213305 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:28.213402 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:28.213504 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 09:29:28.213998 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:28.215259 (+ 1261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:28.217406 (+ 2147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:28.218156 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:28.218308 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:28.218816 (+ 508us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 09:29:28.218956 (+ 140us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 09:29:28.219040 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:28.219220 (+ 180us) server_negotiation.cc:299] Negotiation successful
0806 09:29:28.219301 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":814,"thread_start_us":671,"threads_started":1}
I20260806 09:29:28.223189 32748 raft_consensus.cc:1261] T 9eb5c43e40ef497ea05097c6ad368d22 P d6930c871eec4d2f9c6efa9c8e742cba [term 1 FOLLOWER]: Refusing update from remote peer a27e10c55c654384ae155607041dad72: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 09:29:28.223308 32551 raft_consensus.cc:1261] T 9eb5c43e40ef497ea05097c6ad368d22 P d9485c4273194777b6ccf02c3e050e6f [term 1 FOLLOWER]: Refusing update from remote peer a27e10c55c654384ae155607041dad72: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 09:29:28.223996 409 consensus_queue.cc:1035] T 9eb5c43e40ef497ea05097c6ad368d22 P a27e10c55c654384ae155607041dad72 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6930c871eec4d2f9c6efa9c8e742cba" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 38869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 09:29:28.224690 409 consensus_queue.cc:1035] T 9eb5c43e40ef497ea05097c6ad368d22 P a27e10c55c654384ae155607041dad72 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9485c4273194777b6ccf02c3e050e6f" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 45517 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 09:29:28.249929 32092 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53472:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 09:29:28.250172 32092 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53472:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 09:29:28.253201 32092 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 81679e1de52c40d883ea9f9173070aa7: Sending DeleteTablet for 3 replicas of tablet 9eb5c43e40ef497ea05097c6ad368d22
I20260806 09:29:28.254292 32721 tablet_service.cc:1514] Processing DeleteTablet for tablet 9eb5c43e40ef497ea05097c6ad368d22 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:29:28 UTC) from {username='oryx', principal='oryx/127.20.113.62@KRBTEST.COM'} at 127.0.0.1:50648
I20260806 09:29:28.254606 482 tablet_replica.cc:330] T 9eb5c43e40ef497ea05097c6ad368d22 P d6930c871eec4d2f9c6efa9c8e742cba: stopping tablet replica
I20260806 09:29:28.254815 482 raft_consensus.cc:2229] T 9eb5c43e40ef497ea05097c6ad368d22 P d6930c871eec4d2f9c6efa9c8e742cba [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 09:29:28.254822 32308 tablet_service.cc:1514] Processing DeleteTablet for tablet 9eb5c43e40ef497ea05097c6ad368d22 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:29:28 UTC) from {username='oryx', principal='oryx/127.20.113.62@KRBTEST.COM'} at 127.0.0.1:47208
I20260806 09:29:28.255009 482 raft_consensus.cc:2258] T 9eb5c43e40ef497ea05097c6ad368d22 P d6930c871eec4d2f9c6efa9c8e742cba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 09:29:28.255263 480 tablet_replica.cc:330] T 9eb5c43e40ef497ea05097c6ad368d22 P a27e10c55c654384ae155607041dad72: stopping tablet replica
I20260806 09:29:28.255458 480 raft_consensus.cc:2229] T 9eb5c43e40ef497ea05097c6ad368d22 P a27e10c55c654384ae155607041dad72 [term 1 LEADER]: Raft consensus shutting down.
I20260806 09:29:28.255589 32518 tablet_service.cc:1514] Processing DeleteTablet for tablet 9eb5c43e40ef497ea05097c6ad368d22 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:29:28 UTC) from {username='oryx', principal='oryx/127.20.113.62@KRBTEST.COM'} at 127.0.0.1:52408
I20260806 09:29:28.255789 480 raft_consensus.cc:2258] T 9eb5c43e40ef497ea05097c6ad368d22 P a27e10c55c654384ae155607041dad72 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 09:29:28.256206 481 tablet_replica.cc:330] T 9eb5c43e40ef497ea05097c6ad368d22 P d9485c4273194777b6ccf02c3e050e6f: stopping tablet replica
I20260806 09:29:28.256690 481 raft_consensus.cc:2229] T 9eb5c43e40ef497ea05097c6ad368d22 P d9485c4273194777b6ccf02c3e050e6f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 09:29:28.256946 481 raft_consensus.cc:2258] T 9eb5c43e40ef497ea05097c6ad368d22 P d9485c4273194777b6ccf02c3e050e6f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 09:29:28.256757 32092 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53472:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 09:29:28.257432 32092 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 09:29:28.257899 482 ts_tablet_manager.cc:1905] T 9eb5c43e40ef497ea05097c6ad368d22 P d6930c871eec4d2f9c6efa9c8e742cba: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 09:29:28.258054 480 ts_tablet_manager.cc:1905] T 9eb5c43e40ef497ea05097c6ad368d22 P a27e10c55c654384ae155607041dad72: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 09:29:28.264581 482 ts_tablet_manager.cc:1918] T 9eb5c43e40ef497ea05097c6ad368d22 P d6930c871eec4d2f9c6efa9c8e742cba: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 09:29:28.264747 482 log.cc:1196] T 9eb5c43e40ef497ea05097c6ad368d22 P d6930c871eec4d2f9c6efa9c8e742cba: Deleting WAL directory at /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786008495763054-20932-0/minicluster-data/ts-2/wal/wals/9eb5c43e40ef497ea05097c6ad368d22
I20260806 09:29:28.265205 482 ts_tablet_manager.cc:1939] T 9eb5c43e40ef497ea05097c6ad368d22 P d6930c871eec4d2f9c6efa9c8e742cba: Deleting consensus metadata
I20260806 09:29:28.265676 480 ts_tablet_manager.cc:1918] T 9eb5c43e40ef497ea05097c6ad368d22 P a27e10c55c654384ae155607041dad72: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 09:29:28.265906 480 log.cc:1196] T 9eb5c43e40ef497ea05097c6ad368d22 P a27e10c55c654384ae155607041dad72: Deleting WAL directory at /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786008495763054-20932-0/minicluster-data/ts-0/wal/wals/9eb5c43e40ef497ea05097c6ad368d22
I20260806 09:29:28.266414 480 ts_tablet_manager.cc:1939] T 9eb5c43e40ef497ea05097c6ad368d22 P a27e10c55c654384ae155607041dad72: Deleting consensus metadata
I20260806 09:29:28.268090 32075 catalog_manager.cc:4841] TS a27e10c55c654384ae155607041dad72 (127.20.113.1:33839): tablet 9eb5c43e40ef497ea05097c6ad368d22 (table test-table [id=79343b28556c49ee83207f48190ea096]) successfully deleted
I20260806 09:29:28.272550 32308 tablet_service.cc:1467] Processing CreateTablet for tablet f7b5db22016244c0b9470e65b97c9a67 (DEFAULT_TABLE table=test-table [id=9f8624fd99b64c499488ea1362b61756]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:29:28.272984 32308 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f7b5db22016244c0b9470e65b97c9a67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:29:28.273056 481 ts_tablet_manager.cc:1905] T 9eb5c43e40ef497ea05097c6ad368d22 P d9485c4273194777b6ccf02c3e050e6f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 09:29:28.276849 32518 tablet_service.cc:1467] Processing CreateTablet for tablet f7b5db22016244c0b9470e65b97c9a67 (DEFAULT_TABLE table=test-table [id=9f8624fd99b64c499488ea1362b61756]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:29:28.277333 32518 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f7b5db22016244c0b9470e65b97c9a67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:29:28.280261 481 ts_tablet_manager.cc:1918] T 9eb5c43e40ef497ea05097c6ad368d22 P d9485c4273194777b6ccf02c3e050e6f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 09:29:28.280494 481 log.cc:1196] T 9eb5c43e40ef497ea05097c6ad368d22 P d9485c4273194777b6ccf02c3e050e6f: Deleting WAL directory at /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786008495763054-20932-0/minicluster-data/ts-1/wal/wals/9eb5c43e40ef497ea05097c6ad368d22
I20260806 09:29:28.280735 32075 catalog_manager.cc:4841] TS d6930c871eec4d2f9c6efa9c8e742cba (127.20.113.3:38869): tablet 9eb5c43e40ef497ea05097c6ad368d22 (table test-table [id=79343b28556c49ee83207f48190ea096]) successfully deleted
I20260806 09:29:28.281117 481 ts_tablet_manager.cc:1939] T 9eb5c43e40ef497ea05097c6ad368d22 P d9485c4273194777b6ccf02c3e050e6f: Deleting consensus metadata
I20260806 09:29:28.281581 32721 tablet_service.cc:1467] Processing CreateTablet for tablet f7b5db22016244c0b9470e65b97c9a67 (DEFAULT_TABLE table=test-table [id=9f8624fd99b64c499488ea1362b61756]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:29:28.282085 32721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f7b5db22016244c0b9470e65b97c9a67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:29:28.284526 32077 catalog_manager.cc:4841] TS d9485c4273194777b6ccf02c3e050e6f (127.20.113.2:45517): tablet 9eb5c43e40ef497ea05097c6ad368d22 (table test-table [id=79343b28556c49ee83207f48190ea096]) successfully deleted
I20260806 09:29:28.287686 395 tablet_bootstrap.cc:492] T f7b5db22016244c0b9470e65b97c9a67 P d6930c871eec4d2f9c6efa9c8e742cba: Bootstrap starting.
I20260806 09:29:28.289492 395 tablet_bootstrap.cc:654] T f7b5db22016244c0b9470e65b97c9a67 P d6930c871eec4d2f9c6efa9c8e742cba: Neither blocks nor log segments found. Creating new log.
I20260806 09:29:28.294348 397 tablet_bootstrap.cc:492] T f7b5db22016244c0b9470e65b97c9a67 P a27e10c55c654384ae155607041dad72: Bootstrap starting.
I20260806 09:29:28.296366 396 tablet_bootstrap.cc:492] T f7b5db22016244c0b9470e65b97c9a67 P d9485c4273194777b6ccf02c3e050e6f: Bootstrap starting.
I20260806 09:29:28.296880 397 tablet_bootstrap.cc:654] T f7b5db22016244c0b9470e65b97c9a67 P a27e10c55c654384ae155607041dad72: Neither blocks nor log segments found. Creating new log.
I20260806 09:29:28.298110 396 tablet_bootstrap.cc:654] T f7b5db22016244c0b9470e65b97c9a67 P d9485c4273194777b6ccf02c3e050e6f: Neither blocks nor log segments found. Creating new log.
I20260806 09:29:28.300627 397 tablet_bootstrap.cc:492] T f7b5db22016244c0b9470e65b97c9a67 P a27e10c55c654384ae155607041dad72: No bootstrap required, opened a new log
I20260806 09:29:28.300724 396 tablet_bootstrap.cc:492] T f7b5db22016244c0b9470e65b97c9a67 P d9485c4273194777b6ccf02c3e050e6f: No bootstrap required, opened a new log
I20260806 09:29:28.300809 397 ts_tablet_manager.cc:1397] T f7b5db22016244c0b9470e65b97c9a67 P a27e10c55c654384ae155607041dad72: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260806 09:29:28.300895 396 ts_tablet_manager.cc:1397] T f7b5db22016244c0b9470e65b97c9a67 P d9485c4273194777b6ccf02c3e050e6f: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260806 09:29:28.301616 397 raft_consensus.cc:348] T f7b5db22016244c0b9470e65b97c9a67 P a27e10c55c654384ae155607041dad72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a27e10c55c654384ae155607041dad72" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33839 } } peers { permanent_uuid: "d9485c4273194777b6ccf02c3e050e6f" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 45517 } } peers { permanent_uuid: "d6930c871eec4d2f9c6efa9c8e742cba" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 38869 } }
I20260806 09:29:28.301628 396 raft_consensus.cc:348] T f7b5db22016244c0b9470e65b97c9a67 P d9485c4273194777b6ccf02c3e050e6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a27e10c55c654384ae155607041dad72" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33839 } } peers { permanent_uuid: "d9485c4273194777b6ccf02c3e050e6f" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 45517 } } peers { permanent_uuid: "d6930c871eec4d2f9c6efa9c8e742cba" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 38869 } }
I20260806 09:29:28.301892 397 raft_consensus.cc:374] T f7b5db22016244c0b9470e65b97c9a67 P a27e10c55c654384ae155607041dad72 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:29:28.301911 396 raft_consensus.cc:374] T f7b5db22016244c0b9470e65b97c9a67 P d9485c4273194777b6ccf02c3e050e6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:29:28.301973 397 raft_consensus.cc:729] T f7b5db22016244c0b9470e65b97c9a67 P a27e10c55c654384ae155607041dad72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a27e10c55c654384ae155607041dad72, State: Initialized, Role: FOLLOWER
I20260806 09:29:28.301991 396 raft_consensus.cc:729] T f7b5db22016244c0b9470e65b97c9a67 P d9485c4273194777b6ccf02c3e050e6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9485c4273194777b6ccf02c3e050e6f, State: Initialized, Role: FOLLOWER
I20260806 09:29:28.302205 397 consensus_queue.cc:260] T f7b5db22016244c0b9470e65b97c9a67 P a27e10c55c654384ae155607041dad72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a27e10c55c654384ae155607041dad72" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33839 } } peers { permanent_uuid: "d9485c4273194777b6ccf02c3e050e6f" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 45517 } } peers { permanent_uuid: "d6930c871eec4d2f9c6efa9c8e742cba" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 38869 } }
I20260806 09:29:28.302271 396 consensus_queue.cc:260] T f7b5db22016244c0b9470e65b97c9a67 P d9485c4273194777b6ccf02c3e050e6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a27e10c55c654384ae155607041dad72" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33839 } } peers { permanent_uuid: "d9485c4273194777b6ccf02c3e050e6f" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 45517 } } peers { permanent_uuid: "d6930c871eec4d2f9c6efa9c8e742cba" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 38869 } }
I20260806 09:29:28.302718 397 ts_tablet_manager.cc:1428] T f7b5db22016244c0b9470e65b97c9a67 P a27e10c55c654384ae155607041dad72: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 09:29:28.302834 396 ts_tablet_manager.cc:1428] T f7b5db22016244c0b9470e65b97c9a67 P d9485c4273194777b6ccf02c3e050e6f: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 09:29:28.304090 395 tablet_bootstrap.cc:492] T f7b5db22016244c0b9470e65b97c9a67 P d6930c871eec4d2f9c6efa9c8e742cba: No bootstrap required, opened a new log
I20260806 09:29:28.304273 395 ts_tablet_manager.cc:1397] T f7b5db22016244c0b9470e65b97c9a67 P d6930c871eec4d2f9c6efa9c8e742cba: Time spent bootstrapping tablet: real 0.017s user 0.003s sys 0.008s
I20260806 09:29:28.305516 395 raft_consensus.cc:348] T f7b5db22016244c0b9470e65b97c9a67 P d6930c871eec4d2f9c6efa9c8e742cba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a27e10c55c654384ae155607041dad72" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33839 } } peers { permanent_uuid: "d9485c4273194777b6ccf02c3e050e6f" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 45517 } } peers { permanent_uuid: "d6930c871eec4d2f9c6efa9c8e742cba" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 38869 } }
I20260806 09:29:28.305888 395 raft_consensus.cc:374] T f7b5db22016244c0b9470e65b97c9a67 P d6930c871eec4d2f9c6efa9c8e742cba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:29:28.306012 395 raft_consensus.cc:729] T f7b5db22016244c0b9470e65b97c9a67 P d6930c871eec4d2f9c6efa9c8e742cba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6930c871eec4d2f9c6efa9c8e742cba, State: Initialized, Role: FOLLOWER
I20260806 09:29:28.306272 395 consensus_queue.cc:260] T f7b5db22016244c0b9470e65b97c9a67 P d6930c871eec4d2f9c6efa9c8e742cba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a27e10c55c654384ae155607041dad72" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33839 } } peers { permanent_uuid: "d9485c4273194777b6ccf02c3e050e6f" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 45517 } } peers { permanent_uuid: "d6930c871eec4d2f9c6efa9c8e742cba" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 38869 } }
I20260806 09:29:28.306979 395 ts_tablet_manager.cc:1428] T f7b5db22016244c0b9470e65b97c9a67 P d6930c871eec4d2f9c6efa9c8e742cba: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260806 09:29:28.390712 437 raft_consensus.cc:482] T f7b5db22016244c0b9470e65b97c9a67 P d6930c871eec4d2f9c6efa9c8e742cba [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 09:29:28.390985 437 raft_consensus.cc:504] T f7b5db22016244c0b9470e65b97c9a67 P d6930c871eec4d2f9c6efa9c8e742cba [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a27e10c55c654384ae155607041dad72" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33839 } } peers { permanent_uuid: "d9485c4273194777b6ccf02c3e050e6f" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 45517 } } peers { permanent_uuid: "d6930c871eec4d2f9c6efa9c8e742cba" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 38869 } }
I20260806 09:29:28.391677 437 leader_election.cc:288] T f7b5db22016244c0b9470e65b97c9a67 P d6930c871eec4d2f9c6efa9c8e742cba [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a27e10c55c654384ae155607041dad72 (127.20.113.1:33839), d9485c4273194777b6ccf02c3e050e6f (127.20.113.2:45517)
I20260806 09:29:28.392241 32334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7b5db22016244c0b9470e65b97c9a67" candidate_uuid: "d6930c871eec4d2f9c6efa9c8e742cba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a27e10c55c654384ae155607041dad72" is_pre_election: true
I20260806 09:29:28.392423 32551 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7b5db22016244c0b9470e65b97c9a67" candidate_uuid: "d6930c871eec4d2f9c6efa9c8e742cba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9485c4273194777b6ccf02c3e050e6f" is_pre_election: true
I20260806 09:29:28.392539 32334 raft_consensus.cc:2454] T f7b5db22016244c0b9470e65b97c9a67 P a27e10c55c654384ae155607041dad72 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6930c871eec4d2f9c6efa9c8e742cba in term 0.
I20260806 09:29:28.392722 32551 raft_consensus.cc:2454] T f7b5db22016244c0b9470e65b97c9a67 P d9485c4273194777b6ccf02c3e050e6f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6930c871eec4d2f9c6efa9c8e742cba in term 0.
I20260806 09:29:28.393064 32651 leader_election.cc:302] T f7b5db22016244c0b9470e65b97c9a67 P d6930c871eec4d2f9c6efa9c8e742cba [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a27e10c55c654384ae155607041dad72, d6930c871eec4d2f9c6efa9c8e742cba; no voters:
I20260806 09:29:28.393399 437 raft_consensus.cc:2781] T f7b5db22016244c0b9470e65b97c9a67 P d6930c871eec4d2f9c6efa9c8e742cba [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 09:29:28.393523 437 raft_consensus.cc:482] T f7b5db22016244c0b9470e65b97c9a67 P d6930c871eec4d2f9c6efa9c8e742cba [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 09:29:28.393607 437 raft_consensus.cc:3037] T f7b5db22016244c0b9470e65b97c9a67 P d6930c871eec4d2f9c6efa9c8e742cba [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:29:28.395761 437 raft_consensus.cc:504] T f7b5db22016244c0b9470e65b97c9a67 P d6930c871eec4d2f9c6efa9c8e742cba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a27e10c55c654384ae155607041dad72" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33839 } } peers { permanent_uuid: "d9485c4273194777b6ccf02c3e050e6f" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 45517 } } peers { permanent_uuid: "d6930c871eec4d2f9c6efa9c8e742cba" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 38869 } }
I20260806 09:29:28.396385 437 leader_election.cc:288] T f7b5db22016244c0b9470e65b97c9a67 P d6930c871eec4d2f9c6efa9c8e742cba [CANDIDATE]: Term 1 election: Requested vote from peers a27e10c55c654384ae155607041dad72 (127.20.113.1:33839), d9485c4273194777b6ccf02c3e050e6f (127.20.113.2:45517)
I20260806 09:29:28.396746 32334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7b5db22016244c0b9470e65b97c9a67" candidate_uuid: "d6930c871eec4d2f9c6efa9c8e742cba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a27e10c55c654384ae155607041dad72"
I20260806 09:29:28.396896 32551 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7b5db22016244c0b9470e65b97c9a67" candidate_uuid: "d6930c871eec4d2f9c6efa9c8e742cba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9485c4273194777b6ccf02c3e050e6f"
I20260806 09:29:28.396984 32334 raft_consensus.cc:3037] T f7b5db22016244c0b9470e65b97c9a67 P a27e10c55c654384ae155607041dad72 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:29:28.397106 32551 raft_consensus.cc:3037] T f7b5db22016244c0b9470e65b97c9a67 P d9485c4273194777b6ccf02c3e050e6f [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:29:28.399955 32334 raft_consensus.cc:2454] T f7b5db22016244c0b9470e65b97c9a67 P a27e10c55c654384ae155607041dad72 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6930c871eec4d2f9c6efa9c8e742cba in term 1.
I20260806 09:29:28.400022 32551 raft_consensus.cc:2454] T f7b5db22016244c0b9470e65b97c9a67 P d9485c4273194777b6ccf02c3e050e6f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6930c871eec4d2f9c6efa9c8e742cba in term 1.
I20260806 09:29:28.400411 32653 leader_election.cc:302] T f7b5db22016244c0b9470e65b97c9a67 P d6930c871eec4d2f9c6efa9c8e742cba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d6930c871eec4d2f9c6efa9c8e742cba, d9485c4273194777b6ccf02c3e050e6f; no voters:
I20260806 09:29:28.400679 437 raft_consensus.cc:2781] T f7b5db22016244c0b9470e65b97c9a67 P d6930c871eec4d2f9c6efa9c8e742cba [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:29:28.401214 437 raft_consensus.cc:686] T f7b5db22016244c0b9470e65b97c9a67 P d6930c871eec4d2f9c6efa9c8e742cba [term 1 LEADER]: Becoming Leader. State: Replica: d6930c871eec4d2f9c6efa9c8e742cba, State: Running, Role: LEADER
I20260806 09:29:28.401508 437 consensus_queue.cc:237] T f7b5db22016244c0b9470e65b97c9a67 P d6930c871eec4d2f9c6efa9c8e742cba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a27e10c55c654384ae155607041dad72" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33839 } } peers { permanent_uuid: "d9485c4273194777b6ccf02c3e050e6f" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 45517 } } peers { permanent_uuid: "d6930c871eec4d2f9c6efa9c8e742cba" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 38869 } }
I20260806 09:29:28.404289 32097 catalog_manager.cc:5495] T f7b5db22016244c0b9470e65b97c9a67 P d6930c871eec4d2f9c6efa9c8e742cba reported cstate change: term changed from 0 to 1, leader changed from <none> to d6930c871eec4d2f9c6efa9c8e742cba (127.20.113.3). New cstate: current_term: 1 leader_uuid: "d6930c871eec4d2f9c6efa9c8e742cba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a27e10c55c654384ae155607041dad72" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33839 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9485c4273194777b6ccf02c3e050e6f" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 45517 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6930c871eec4d2f9c6efa9c8e742cba" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 38869 } health_report { overall_health: HEALTHY } } }
I20260806 09:29:28.430177 522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:28.425241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50674 (local address 127.20.113.3:38869)
0806 09:29:28.425838 (+ 597us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:28.425843 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:28.425858 (+ 15us) server_negotiation.cc:407] Connection header received
0806 09:29:28.425909 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:28.425913 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:28.425973 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:28.426054 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 09:29:28.426512 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:28.427270 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:28.428614 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:28.429026 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:28.429119 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:28.429561 (+ 442us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 09:29:28.429662 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 09:29:28.429729 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:28.429859 (+ 130us) server_negotiation.cc:299] Negotiation successful
0806 09:29:28.429917 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":506,"thread_start_us":360,"threads_started":1}
I20260806 09:29:28.438558 32551 raft_consensus.cc:1261] T f7b5db22016244c0b9470e65b97c9a67 P d9485c4273194777b6ccf02c3e050e6f [term 1 FOLLOWER]: Refusing update from remote peer d6930c871eec4d2f9c6efa9c8e742cba: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 09:29:28.438737 32334 raft_consensus.cc:1261] T f7b5db22016244c0b9470e65b97c9a67 P a27e10c55c654384ae155607041dad72 [term 1 FOLLOWER]: Refusing update from remote peer d6930c871eec4d2f9c6efa9c8e742cba: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 09:29:28.439122 518 consensus_queue.cc:1035] T f7b5db22016244c0b9470e65b97c9a67 P d6930c871eec4d2f9c6efa9c8e742cba [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9485c4273194777b6ccf02c3e050e6f" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 45517 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 09:29:28.439507 437 consensus_queue.cc:1035] T f7b5db22016244c0b9470e65b97c9a67 P d6930c871eec4d2f9c6efa9c8e742cba [LEADER]: Connected to new peer: Peer: permanent_uuid: "a27e10c55c654384ae155607041dad72" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 09:29:28.493546 32097 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53472:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 09:29:28.493852 32097 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53472:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 09:29:28.496858 32097 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 81679e1de52c40d883ea9f9173070aa7: Sending DeleteTablet for 3 replicas of tablet f7b5db22016244c0b9470e65b97c9a67
I20260806 09:29:28.497836 32721 tablet_service.cc:1514] Processing DeleteTablet for tablet f7b5db22016244c0b9470e65b97c9a67 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:29:28 UTC) from {username='oryx', principal='oryx/127.20.113.62@KRBTEST.COM'} at 127.0.0.1:50648
I20260806 09:29:28.497836 32308 tablet_service.cc:1514] Processing DeleteTablet for tablet f7b5db22016244c0b9470e65b97c9a67 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:29:28 UTC) from {username='oryx', principal='oryx/127.20.113.62@KRBTEST.COM'} at 127.0.0.1:47208
I20260806 09:29:28.497960 32518 tablet_service.cc:1514] Processing DeleteTablet for tablet f7b5db22016244c0b9470e65b97c9a67 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:29:28 UTC) from {username='oryx', principal='oryx/127.20.113.62@KRBTEST.COM'} at 127.0.0.1:52408
I20260806 09:29:28.498157 480 tablet_replica.cc:330] T f7b5db22016244c0b9470e65b97c9a67 P a27e10c55c654384ae155607041dad72: stopping tablet replica
I20260806 09:29:28.498157 482 tablet_replica.cc:330] T f7b5db22016244c0b9470e65b97c9a67 P d6930c871eec4d2f9c6efa9c8e742cba: stopping tablet replica
I20260806 09:29:28.498294 481 tablet_replica.cc:330] T f7b5db22016244c0b9470e65b97c9a67 P d9485c4273194777b6ccf02c3e050e6f: stopping tablet replica
I20260806 09:29:28.498353 480 raft_consensus.cc:2229] T f7b5db22016244c0b9470e65b97c9a67 P a27e10c55c654384ae155607041dad72 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 09:29:28.498353 482 raft_consensus.cc:2229] T f7b5db22016244c0b9470e65b97c9a67 P d6930c871eec4d2f9c6efa9c8e742cba [term 1 LEADER]: Raft consensus shutting down.
I20260806 09:29:28.498533 481 raft_consensus.cc:2229] T f7b5db22016244c0b9470e65b97c9a67 P d9485c4273194777b6ccf02c3e050e6f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 09:29:28.498569 480 raft_consensus.cc:2258] T f7b5db22016244c0b9470e65b97c9a67 P a27e10c55c654384ae155607041dad72 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 09:29:28.498718 482 raft_consensus.cc:2258] T f7b5db22016244c0b9470e65b97c9a67 P d6930c871eec4d2f9c6efa9c8e742cba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 09:29:28.498749 481 raft_consensus.cc:2258] T f7b5db22016244c0b9470e65b97c9a67 P d9485c4273194777b6ccf02c3e050e6f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 09:29:28.499358 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 32209
I20260806 09:29:28.500487 481 ts_tablet_manager.cc:1905] T f7b5db22016244c0b9470e65b97c9a67 P d9485c4273194777b6ccf02c3e050e6f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 09:29:28.500607 480 ts_tablet_manager.cc:1905] T f7b5db22016244c0b9470e65b97c9a67 P a27e10c55c654384ae155607041dad72: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 09:29:28.505280 482 ts_tablet_manager.cc:1905] T f7b5db22016244c0b9470e65b97c9a67 P d6930c871eec4d2f9c6efa9c8e742cba: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 09:29:28.507582 481 ts_tablet_manager.cc:1918] T f7b5db22016244c0b9470e65b97c9a67 P d9485c4273194777b6ccf02c3e050e6f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 09:29:28.507728 481 log.cc:1196] T f7b5db22016244c0b9470e65b97c9a67 P d9485c4273194777b6ccf02c3e050e6f: Deleting WAL directory at /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786008495763054-20932-0/minicluster-data/ts-1/wal/wals/f7b5db22016244c0b9470e65b97c9a67
I20260806 09:29:28.508185 481 ts_tablet_manager.cc:1939] T f7b5db22016244c0b9470e65b97c9a67 P d9485c4273194777b6ccf02c3e050e6f: Deleting consensus metadata
I20260806 09:29:28.509196 480 ts_tablet_manager.cc:1918] T f7b5db22016244c0b9470e65b97c9a67 P a27e10c55c654384ae155607041dad72: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 09:29:28.509416 480 log.cc:1196] T f7b5db22016244c0b9470e65b97c9a67 P a27e10c55c654384ae155607041dad72: Deleting WAL directory at /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786008495763054-20932-0/minicluster-data/ts-0/wal/wals/f7b5db22016244c0b9470e65b97c9a67
I20260806 09:29:28.509404 32077 catalog_manager.cc:4841] TS d9485c4273194777b6ccf02c3e050e6f (127.20.113.2:45517): tablet f7b5db22016244c0b9470e65b97c9a67 (table test-table [id=9f8624fd99b64c499488ea1362b61756]) successfully deleted
I20260806 09:29:28.509979 480 ts_tablet_manager.cc:1939] T f7b5db22016244c0b9470e65b97c9a67 P a27e10c55c654384ae155607041dad72: Deleting consensus metadata
I20260806 09:29:28.511109 32075 catalog_manager.cc:4841] TS a27e10c55c654384ae155607041dad72 (127.20.113.1:33839): tablet f7b5db22016244c0b9470e65b97c9a67 (table test-table [id=9f8624fd99b64c499488ea1362b61756]) successfully deleted
I20260806 09:29:28.512025 482 ts_tablet_manager.cc:1918] T f7b5db22016244c0b9470e65b97c9a67 P d6930c871eec4d2f9c6efa9c8e742cba: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 09:29:28.512187 482 log.cc:1196] T f7b5db22016244c0b9470e65b97c9a67 P d6930c871eec4d2f9c6efa9c8e742cba: Deleting WAL directory at /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786008495763054-20932-0/minicluster-data/ts-2/wal/wals/f7b5db22016244c0b9470e65b97c9a67
I20260806 09:29:28.512612 482 ts_tablet_manager.cc:1939] T f7b5db22016244c0b9470e65b97c9a67 P d6930c871eec4d2f9c6efa9c8e742cba: Deleting consensus metadata
I20260806 09:29:28.513661 32075 catalog_manager.cc:4841] TS d6930c871eec4d2f9c6efa9c8e742cba (127.20.113.3:38869): tablet f7b5db22016244c0b9470e65b97c9a67 (table test-table [id=9f8624fd99b64c499488ea1362b61756]) successfully deleted
Aug 06 09:29:28 dist-test-slave-45dx krb5kdc[32029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008567, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.20.113.1@KRBTEST.COM
I20260806 09:29:28.521704 509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:28.504995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47242 (local address 127.20.113.1:33839)
0806 09:29:28.505147 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:28.505151 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:28.505168 (+ 17us) server_negotiation.cc:407] Connection header received
0806 09:29:28.505389 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:28.505396 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:28.505496 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:28.505597 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:28.507449 (+ 1852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:28.508643 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:28.510589 (+ 1946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:28.511285 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:28.511422 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:28.516181 (+ 4759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:29:28.516201 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:29:28.516217 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:29:28.516260 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:29:28.519172 (+ 2912us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:28.519845 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:28.519852 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:28.519854 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:28.519979 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:28.520356 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:28.520361 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:28.520364 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:28.520726 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:29:28.521001 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:28.521279 (+ 278us) server_negotiation.cc:299] Negotiation successful
0806 09:29:28.521463 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 09:29:28.522502 32403 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:29:28.674152 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 32209
I20260806 09:29:28.711010 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 32418
Aug 06 09:29:28 dist-test-slave-45dx krb5kdc[32029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008567, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.20.113.2@KRBTEST.COM
I20260806 09:29:28.726392 538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:28.711848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52424 (local address 127.20.113.2:45517)
0806 09:29:28.712416 (+ 568us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:28.712421 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:28.712446 (+ 25us) server_negotiation.cc:407] Connection header received
0806 09:29:28.712504 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:28.712508 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:28.712567 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:28.712661 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:28.713874 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:28.714643 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:28.715820 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:28.716251 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:28.716427 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:28.720376 (+ 3949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:29:28.720391 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:29:28.720407 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:29:28.720449 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:29:28.723746 (+ 3297us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:28.724490 (+ 744us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:28.724502 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:28.724507 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:28.724653 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:28.725003 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:28.725007 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:28.725009 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:28.725351 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:29:28.725586 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:28.725841 (+ 255us) server_negotiation.cc:299] Negotiation successful
0806 09:29:28.726007 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":382,"thread_start_us":264,"threads_started":1}
I20260806 09:29:28.727080 32618 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:29:28.827031 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 32418
I20260806 09:29:28.853572 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 32631
Aug 06 09:29:28 dist-test-slave-45dx krb5kdc[32029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008567, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.20.113.3@KRBTEST.COM
I20260806 09:29:28.868673 522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:28.854373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50686 (local address 127.20.113.3:38869)
0806 09:29:28.854572 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:28.854577 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:28.854595 (+ 18us) server_negotiation.cc:407] Connection header received
0806 09:29:28.854739 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:28.854743 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:28.854818 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:28.854907 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:28.856193 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:28.857028 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:28.858304 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:28.858762 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:28.858853 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:28.863408 (+ 4555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:29:28.863423 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:29:28.863450 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:29:28.863483 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:29:28.866318 (+ 2835us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:28.866944 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:28.866951 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:28.866953 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:28.867078 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:28.867393 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:28.867396 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:28.867398 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:28.867767 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:29:28.867998 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:28.868185 (+ 187us) server_negotiation.cc:299] Negotiation successful
0806 09:29:28.868416 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 09:29:28.869371 343 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:29:28.994756 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 32631
I20260806 09:29:29.025707 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 32056
I20260806 09:29:29.026926 32164 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:29:29.146145 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 32056
2026-08-06T09:29:29Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (7161 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786008495763054-20932-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 09:29:29 dist-test-slave-45dx krb5kdc[548](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 09:29:29 dist-test-slave-45dx krb5kdc[548](info): set up 2 sockets
Aug 06 09:29:29 dist-test-slave-45dx krb5kdc[548](info): commencing operation
krb5kdc: starting...
W20260806 09:29:30.471601 20932 mini_kdc.cc:121] Time spent starting KDC: real 1.269s user 0.004s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786008495763054-20932-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786008495763054-20932-0/krb5kdc/test-user.keytab.
Aug 06 09:29:30 dist-test-slave-45dx krb5kdc[548](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008570, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T09:29:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T09:29:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.113.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.113.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.113.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab.
Entry for principal kudu/127.20.113.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab.
I20260806 09:29:30.759443 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786008495763054-20932-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786008495763054-20932-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786008495763054-20932-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.113.62:40977
--webserver_interface=127.20.113.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab
--principal=kudu/127.20.113.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:34169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.113.62:40977
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:29:31.119926 565 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:29:31.120214 565 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:29:31.120397 565 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:29:31.127863 565 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 09:29:31.128026 565 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:29:31.128118 565 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:29:31.128224 565 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 09:29:31.128325 565 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 09:29:31.138655 565 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:34169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786008495763054-20932-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786008495763054-20932-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.113.62:40977
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.113.62:40977
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.113.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786008495763054-20932-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:29:31.140062 565 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:29:31.141866 565 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:29:31.154577 574 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:29:31.154654 571 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:29:31.154676 572 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:29:31.155526 565 server_base.cc:1034] running on GCE node
I20260806 09:29:31.156286 565 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:29:31.157722 565 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:29:31.158924 565 hybrid_clock.cc:648] HybridClock initialized: now 1786008571158860 us; error 56 us; skew 500 ppm
I20260806 09:29:31.159209 565 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-06T09:29:31Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (2074 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 09:29:31 dist-test-slave-45dx krb5kdc[581](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 09:29:31 dist-test-slave-45dx krb5kdc[581](info): set up 2 sockets
Aug 06 09:29:31 dist-test-slave-45dx krb5kdc[581](info): commencing operation
krb5kdc: starting...
W20260806 09:29:32.525146 20932 mini_kdc.cc:121] Time spent starting KDC: real 1.248s user 0.010s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/krb5kdc/test-user.keytab.
Aug 06 09:29:32 dist-test-slave-45dx krb5kdc[581](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008572, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T09:29:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T09:29:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.113.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.113.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.113.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab.
Entry for principal kudu/127.20.113.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab.
I20260806 09:29:32.804564 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.113.62:44663
--webserver_interface=127.20.113.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab
--principal=kudu/127.20.113.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:39947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.113.62:44663
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:29:33.143244 608 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:29:33.143597 608 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:29:33.143752 608 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:29:33.151158 608 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 09:29:33.151314 608 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:29:33.151422 608 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:29:33.151538 608 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 09:29:33.151610 608 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 09:29:33.161855 608 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:39947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.113.62:44663
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab
--principal=kudu/127.20.113.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.113.62:44663
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.113.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:29:33.163189 608 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:29:33.164541 608 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:29:33.176532 614 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:29:33.176581 615 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:29:33.176606 617 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:29:33.177685 608 server_base.cc:1034] running on GCE node
I20260806 09:29:33.178468 608 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:29:33.179795 608 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:29:33.181035 608 hybrid_clock.cc:648] HybridClock initialized: now 1786008573181038 us; error 98 us; skew 500 ppm
I20260806 09:29:33.181387 608 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:29:33 dist-test-slave-45dx krb5kdc[581](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008573, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:29:33.190747 608 init.cc:377] Logged in from keytab as kudu/127.20.113.62@KRBTEST.COM (short username kudu)
I20260806 09:29:33.192284 608 webserver.cc:460] Webserver started at http://127.20.113.62:44423/ using document root <none> and password file <none>
I20260806 09:29:33.193073 608 fs_manager.cc:359] Metadata directory not provided
I20260806 09:29:33.193226 608 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:29:33.193554 608 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:29:33.196599 608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/master-0/data/instance:
uuid: "35c86c26c9844d93a4b84c16b1682362"
format_stamp: "Formatted at 2026-08-06 09:29:33 on dist-test-slave-45dx"
server_key: "798dbae39d26d73b98b5475cd8b37088"
server_key_iv: "1ddcfdb7ef4743275817f0a2663fb05e"
server_key_version: "encryptionkey@0"
I20260806 09:29:33.197392 608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/master-0/wal/instance:
uuid: "35c86c26c9844d93a4b84c16b1682362"
format_stamp: "Formatted at 2026-08-06 09:29:33 on dist-test-slave-45dx"
server_key: "798dbae39d26d73b98b5475cd8b37088"
server_key_iv: "1ddcfdb7ef4743275817f0a2663fb05e"
server_key_version: "encryptionkey@0"
I20260806 09:29:33.202610 608 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260806 09:29:33.206050 624 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:29:33.207350 608 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 09:29:33.207549 608 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/master-0/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/master-0/wal
uuid: "35c86c26c9844d93a4b84c16b1682362"
format_stamp: "Formatted at 2026-08-06 09:29:33 on dist-test-slave-45dx"
server_key: "798dbae39d26d73b98b5475cd8b37088"
server_key_iv: "1ddcfdb7ef4743275817f0a2663fb05e"
server_key_version: "encryptionkey@0"
I20260806 09:29:33.207742 608 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:29:33.218636 608 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:29:33.222500 608 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:29:33.222805 608 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:29:33.241623 608 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.62:44663
I20260806 09:29:33.241648 692 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.62:44663 every 8 connection(s)
I20260806 09:29:33.243129 608 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
I20260806 09:29:33.247058 694 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:29:33.251821 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 608
I20260806 09:29:33.252192 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/master-0/wal/instance
I20260806 09:29:33.253089 20932 external_mini_cluster.cc:1428] Setting key 53a790c9b70cfd11b29f6d76f2995aa2
I20260806 09:29:33.259194 694 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 35c86c26c9844d93a4b84c16b1682362: Bootstrap starting.
I20260806 09:29:33.263023 694 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 35c86c26c9844d93a4b84c16b1682362: Neither blocks nor log segments found. Creating new log.
I20260806 09:29:33.264213 694 log.cc:824] T 00000000000000000000000000000000 P 35c86c26c9844d93a4b84c16b1682362: Log is configured to *not* fsync() on all Append() calls
I20260806 09:29:33.267767 694 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 35c86c26c9844d93a4b84c16b1682362: No bootstrap required, opened a new log
Aug 06 09:29:33 dist-test-slave-45dx krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008572, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.113.62@KRBTEST.COM
I20260806 09:29:33.278079 697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:33.255234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50470 (local address 127.20.113.62:44663)
0806 09:29:33.256220 (+ 986us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:33.256231 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:33.256272 (+ 41us) server_negotiation.cc:407] Connection header received
0806 09:29:33.258048 (+ 1776us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:33.258070 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:33.258414 (+ 344us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:33.258729 (+ 315us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:33.260633 (+ 1904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:33.262122 (+ 1489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:33.263866 (+ 1744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:33.264616 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:33.264835 (+ 219us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:33.269554 (+ 4719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:29:33.269564 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:29:33.269582 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:29:33.269628 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:29:33.273610 (+ 3982us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:33.274300 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:33.274308 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:33.274318 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:33.274540 (+ 222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:33.274894 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:33.274899 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:33.274902 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:33.275592 (+ 690us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:29:33.275897 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:33.276253 (+ 356us) server_negotiation.cc:299] Negotiation successful
0806 09:29:33.276736 (+ 483us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":762,"thread_start_us":537,"threads_started":1}
I20260806 09:29:33.278580 694 raft_consensus.cc:348] T 00000000000000000000000000000000 P 35c86c26c9844d93a4b84c16b1682362 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35c86c26c9844d93a4b84c16b1682362" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 44663 } }
I20260806 09:29:33.278972 694 raft_consensus.cc:374] T 00000000000000000000000000000000 P 35c86c26c9844d93a4b84c16b1682362 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:29:33.279093 694 raft_consensus.cc:729] T 00000000000000000000000000000000 P 35c86c26c9844d93a4b84c16b1682362 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35c86c26c9844d93a4b84c16b1682362, State: Initialized, Role: FOLLOWER
I20260806 09:29:33.279639 694 consensus_queue.cc:260] T 00000000000000000000000000000000 P 35c86c26c9844d93a4b84c16b1682362 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35c86c26c9844d93a4b84c16b1682362" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 44663 } }
I20260806 09:29:33.279943 694 raft_consensus.cc:388] T 00000000000000000000000000000000 P 35c86c26c9844d93a4b84c16b1682362 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 09:29:33.280097 694 raft_consensus.cc:482] T 00000000000000000000000000000000 P 35c86c26c9844d93a4b84c16b1682362 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 09:29:33.280284 694 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 35c86c26c9844d93a4b84c16b1682362 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:29:33.282663 694 raft_consensus.cc:504] T 00000000000000000000000000000000 P 35c86c26c9844d93a4b84c16b1682362 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35c86c26c9844d93a4b84c16b1682362" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 44663 } }
I20260806 09:29:33.283213 694 leader_election.cc:302] T 00000000000000000000000000000000 P 35c86c26c9844d93a4b84c16b1682362 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 35c86c26c9844d93a4b84c16b1682362; no voters:
I20260806 09:29:33.283808 694 leader_election.cc:288] T 00000000000000000000000000000000 P 35c86c26c9844d93a4b84c16b1682362 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 09:29:33.283910 700 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 35c86c26c9844d93a4b84c16b1682362 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:29:33.284740 700 raft_consensus.cc:686] T 00000000000000000000000000000000 P 35c86c26c9844d93a4b84c16b1682362 [term 1 LEADER]: Becoming Leader. State: Replica: 35c86c26c9844d93a4b84c16b1682362, State: Running, Role: LEADER
I20260806 09:29:33.285306 700 consensus_queue.cc:237] T 00000000000000000000000000000000 P 35c86c26c9844d93a4b84c16b1682362 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35c86c26c9844d93a4b84c16b1682362" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 44663 } }
I20260806 09:29:33.286087 694 sys_catalog.cc:564] T 00000000000000000000000000000000 P 35c86c26c9844d93a4b84c16b1682362 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 09:29:33.288414 702 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35c86c26c9844d93a4b84c16b1682362 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "35c86c26c9844d93a4b84c16b1682362" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35c86c26c9844d93a4b84c16b1682362" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 44663 } } }
I20260806 09:29:33.288765 702 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35c86c26c9844d93a4b84c16b1682362 [sys.catalog]: This master's current role is: LEADER
I20260806 09:29:33.289256 704 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35c86c26c9844d93a4b84c16b1682362 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 35c86c26c9844d93a4b84c16b1682362. Latest consensus state: current_term: 1 leader_uuid: "35c86c26c9844d93a4b84c16b1682362" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35c86c26c9844d93a4b84c16b1682362" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 44663 } } }
I20260806 09:29:33.289587 704 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35c86c26c9844d93a4b84c16b1682362 [sys.catalog]: This master's current role is: LEADER
I20260806 09:29:33.290808 712 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 09:29:33.296269 712 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 09:29:33.305408 712 catalog_manager.cc:1347] Generated new cluster ID: 35b03f9873fe493db120b6047fec8f3b
I20260806 09:29:33.305536 712 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 09:29:33.334415 712 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 09:29:33.336395 712 catalog_manager.cc:1504] Loading token signing keys...
I20260806 09:29:33.346138 712 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 35c86c26c9844d93a4b84c16b1682362: Generated new TSK 0
I20260806 09:29:33.347209 712 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.113.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.113.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.113.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.1.keytab.
Entry for principal kudu/127.20.113.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.1.keytab.
I20260806 09:29:33.424089 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.113.1:0
--local_ip_for_outbound_sockets=127.20.113.1
--webserver_interface=127.20.113.1
--webserver_port=0
--tserver_master_addrs=127.20.113.62:44663
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.1.keytab
--principal=kudu/127.20.113.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:39947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:29:33.769711 737 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:29:33.769989 737 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:29:33.770081 737 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:29:33.777315 737 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:29:33.777464 737 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:29:33.777634 737 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.113.1
I20260806 09:29:33.787729 737 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:39947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.1.keytab
--principal=kudu/127.20.113.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.113.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.113.1
--webserver_port=0
--tserver_master_addrs=127.20.113.62:44663
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.113.1
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:29:33.789098 737 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:29:33.790444 737 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:29:33.803808 744 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:29:33.803876 745 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:29:33.803905 747 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:29:33.804998 737 server_base.cc:1034] running on GCE node
I20260806 09:29:33.805558 737 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:29:33.806396 737 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:29:33.807587 737 hybrid_clock.cc:648] HybridClock initialized: now 1786008573807546 us; error 31 us; skew 500 ppm
I20260806 09:29:33.807852 737 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:29:33 dist-test-slave-45dx krb5kdc[581](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008573, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:29:33.810937 737 init.cc:377] Logged in from keytab as kudu/127.20.113.1@KRBTEST.COM (short username kudu)
I20260806 09:29:33.812563 737 webserver.cc:460] Webserver started at http://127.20.113.1:36519/ using document root <none> and password file <none>
I20260806 09:29:33.813269 737 fs_manager.cc:359] Metadata directory not provided
I20260806 09:29:33.813437 737 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:29:33.813773 737 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:29:33.816726 737 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/ts-0/data/instance:
uuid: "63687bd3556c4de6be5505108c3dac7e"
format_stamp: "Formatted at 2026-08-06 09:29:33 on dist-test-slave-45dx"
server_key: "bd56a92f73790bd50124833fb4320936"
server_key_iv: "47297d235f0cb7d920c91f9506b43ec0"
server_key_version: "encryptionkey@0"
I20260806 09:29:33.817526 737 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/ts-0/wal/instance:
uuid: "63687bd3556c4de6be5505108c3dac7e"
format_stamp: "Formatted at 2026-08-06 09:29:33 on dist-test-slave-45dx"
server_key: "bd56a92f73790bd50124833fb4320936"
server_key_iv: "47297d235f0cb7d920c91f9506b43ec0"
server_key_version: "encryptionkey@0"
I20260806 09:29:33.822575 737 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260806 09:29:33.825965 754 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:29:33.827205 737 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 09:29:33.827396 737 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/ts-0/wal
uuid: "63687bd3556c4de6be5505108c3dac7e"
format_stamp: "Formatted at 2026-08-06 09:29:33 on dist-test-slave-45dx"
server_key: "bd56a92f73790bd50124833fb4320936"
server_key_iv: "47297d235f0cb7d920c91f9506b43ec0"
server_key_version: "encryptionkey@0"
I20260806 09:29:33.827597 737 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:29:33.843565 737 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:29:33.848114 737 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:29:33.848459 737 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:29:33.849575 737 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:29:33.850927 737 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:29:33.851011 737 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:29:33.851182 737 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:29:33.851249 737 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:29:33.883509 737 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.1:46421
I20260806 09:29:33.883545 901 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.1:46421 every 8 connection(s)
I20260806 09:29:33.884991 737 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/ts-0/data/info.pb
I20260806 09:29:33.893160 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 737
I20260806 09:29:33.893399 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/ts-0/wal/instance
I20260806 09:29:33.894415 20932 external_mini_cluster.cc:1428] Setting key 977c8305595321ff2b0ea9159e18231c
Aug 06 09:29:33 dist-test-slave-45dx krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008573, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.1@KRBTEST.COM for kudu/127.20.113.62@KRBTEST.COM
I20260806 09:29:33.907214 905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:33.887791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.1:33767 (local address 127.20.113.62:44663)
0806 09:29:33.888242 (+ 451us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:33.888247 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:33.889215 (+ 968us) server_negotiation.cc:407] Connection header received
0806 09:29:33.891154 (+ 1939us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:33.891160 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:33.891247 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:33.891378 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:33.893202 (+ 1824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:33.894073 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:33.896224 (+ 2151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:33.896821 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:33.896926 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:33.901468 (+ 4542us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:29:33.901473 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:29:33.901476 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:29:33.901509 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:29:33.904043 (+ 2534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:33.904853 (+ 810us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:33.904857 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:33.904859 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:33.904976 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:33.905487 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:33.905491 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:33.905493 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:33.905741 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:29:33.905910 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:33.906616 (+ 706us) server_negotiation.cc:299] Negotiation successful
0806 09:29:33.906872 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":231,"threads_started":1}
I20260806 09:29:33.908262 906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:33.888343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44663 (local address 127.20.113.1:33767)
0806 09:29:33.889052 (+ 709us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:33.889089 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:33.890878 (+ 1789us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:33.891677 (+ 799us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:33.891693 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:33.892215 (+ 522us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:29:33.893009 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:33.893039 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:33.894332 (+ 1293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:33.894344 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:33.896080 (+ 1736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:33.896093 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:33.897064 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:33.897071 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:33.897396 (+ 325us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:33.898217 (+ 821us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:29:33.898252 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:29:33.901225 (+ 2973us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:29:33.904201 (+ 2976us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:29:33.904209 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:29:33.904223 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:29:33.904708 (+ 485us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:29:33.905120 (+ 412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:29:33.905124 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:29:33.905128 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:29:33.905353 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:29:33.905882 (+ 529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:29:33.905890 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:29:33.906215 (+ 325us) client_negotiation.cc:769] Sending connection context
0806 09:29:33.906487 (+ 272us) client_negotiation.cc:240] Negotiation successful
0806 09:29:33.906998 (+ 511us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":256,"threads_started":1}
I20260806 09:29:33.909951 903 heartbeater.cc:347] Connected to a master server at 127.20.113.62:44663
I20260806 09:29:33.910251 903 heartbeater.cc:464] Registering TS with master...
I20260806 09:29:33.910996 903 heartbeater.cc:511] Master 127.20.113.62:44663 requested a full tablet report, sending...
I20260806 09:29:33.913193 646 ts_manager.cc:194] Registered new tserver with Master: 63687bd3556c4de6be5505108c3dac7e (127.20.113.1:46421)
I20260806 09:29:33.914552 646 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.113.1@KRBTEST.COM'} at 127.20.113.1:33767
WARNING: no policy specified for kudu/127.20.113.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.113.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.113.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.2.keytab.
Entry for principal kudu/127.20.113.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.2.keytab.
I20260806 09:29:33.961407 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.113.2:0
--local_ip_for_outbound_sockets=127.20.113.2
--webserver_interface=127.20.113.2
--webserver_port=0
--tserver_master_addrs=127.20.113.62:44663
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.2.keytab
--principal=kudu/127.20.113.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:39947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:29:34.311100 914 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:29:34.311452 914 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:29:34.311633 914 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:29:34.318881 914 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:29:34.318974 914 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:29:34.319094 914 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.113.2
I20260806 09:29:34.328526 914 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:39947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.2.keytab
--principal=kudu/127.20.113.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.113.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.113.2
--webserver_port=0
--tserver_master_addrs=127.20.113.62:44663
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.113.2
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:29:34.329872 914 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:29:34.331274 914 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:29:34.345043 922 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:29:34.345008 923 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:29:34.345088 926 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:29:34.345925 914 server_base.cc:1034] running on GCE node
I20260806 09:29:34.346532 914 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:29:34.347450 914 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:29:34.348695 914 hybrid_clock.cc:648] HybridClock initialized: now 1786008574348650 us; error 57 us; skew 500 ppm
I20260806 09:29:34.349022 914 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:29:34 dist-test-slave-45dx krb5kdc[581](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008574, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:29:34.352414 914 init.cc:377] Logged in from keytab as kudu/127.20.113.2@KRBTEST.COM (short username kudu)
I20260806 09:29:34.353773 914 webserver.cc:460] Webserver started at http://127.20.113.2:36013/ using document root <none> and password file <none>
I20260806 09:29:34.354451 914 fs_manager.cc:359] Metadata directory not provided
I20260806 09:29:34.354597 914 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:29:34.354930 914 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:29:34.357872 914 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/ts-1/data/instance:
uuid: "f88a30a2bfad40dab735bdcad75449c2"
format_stamp: "Formatted at 2026-08-06 09:29:34 on dist-test-slave-45dx"
server_key: "ce4d6ae9c0cd92a155caeadeb0c02bfe"
server_key_iv: "c584945e830e2d8cac72f292d420a1e2"
server_key_version: "encryptionkey@0"
I20260806 09:29:34.358671 914 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/ts-1/wal/instance:
uuid: "f88a30a2bfad40dab735bdcad75449c2"
format_stamp: "Formatted at 2026-08-06 09:29:34 on dist-test-slave-45dx"
server_key: "ce4d6ae9c0cd92a155caeadeb0c02bfe"
server_key_iv: "c584945e830e2d8cac72f292d420a1e2"
server_key_version: "encryptionkey@0"
I20260806 09:29:34.363771 914 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260806 09:29:34.367132 936 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:29:34.368626 914 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260806 09:29:34.368813 914 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/ts-1/wal
uuid: "f88a30a2bfad40dab735bdcad75449c2"
format_stamp: "Formatted at 2026-08-06 09:29:34 on dist-test-slave-45dx"
server_key: "ce4d6ae9c0cd92a155caeadeb0c02bfe"
server_key_iv: "c584945e830e2d8cac72f292d420a1e2"
server_key_version: "encryptionkey@0"
I20260806 09:29:34.369035 914 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:29:34.408545 914 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:29:34.413282 914 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:29:34.413538 914 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:29:34.414443 914 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:29:34.415755 914 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:29:34.415831 914 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:29:34.415943 914 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:29:34.415989 914 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:29:34.450824 914 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.2:40769
I20260806 09:29:34.450842 1087 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.2:40769 every 8 connection(s)
I20260806 09:29:34.452291 914 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/ts-1/data/info.pb
I20260806 09:29:34.461201 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 914
I20260806 09:29:34.461380 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/ts-1/wal/instance
I20260806 09:29:34.461867 20932 external_mini_cluster.cc:1428] Setting key e46740c3eae7b88b7fe0c0f49aea01d4
Aug 06 09:29:34 dist-test-slave-45dx krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008574, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.2@KRBTEST.COM for kudu/127.20.113.62@KRBTEST.COM
I20260806 09:29:34.473316 1091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:34.455052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.2:39279 (local address 127.20.113.62:44663)
0806 09:29:34.455516 (+ 464us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:34.455520 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:34.456475 (+ 955us) server_negotiation.cc:407] Connection header received
0806 09:29:34.458374 (+ 1899us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:34.458382 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:34.458485 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:34.458607 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:34.460157 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:34.460930 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:34.462724 (+ 1794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:34.463283 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:34.463387 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:34.467163 (+ 3776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:29:34.467168 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:29:34.467171 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:29:34.467203 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:29:34.470127 (+ 2924us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:34.470849 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:34.470856 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:34.470859 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:34.471030 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:34.471459 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:34.471466 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:34.471469 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:34.471745 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:29:34.471919 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:34.472684 (+ 765us) server_negotiation.cc:299] Negotiation successful
0806 09:29:34.472942 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":248,"threads_started":1}
I20260806 09:29:34.474350 1092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:34.455502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44663 (local address 127.20.113.2:39279)
0806 09:29:34.456245 (+ 743us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:34.456285 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:34.458097 (+ 1812us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:34.458865 (+ 768us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:34.458875 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:34.459419 (+ 544us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:29:34.459986 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:34.460003 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:34.461084 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:34.461089 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:34.462532 (+ 1443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:34.462548 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:34.463537 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:34.463545 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:34.463878 (+ 333us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:34.464737 (+ 859us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:29:34.464760 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:29:34.466935 (+ 2175us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:29:34.470267 (+ 3332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:29:34.470277 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:29:34.470293 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:29:34.470728 (+ 435us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:29:34.471148 (+ 420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:29:34.471154 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:29:34.471157 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:29:34.471343 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:29:34.471887 (+ 544us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:29:34.471896 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:29:34.472243 (+ 347us) client_negotiation.cc:769] Sending connection context
0806 09:29:34.472589 (+ 346us) client_negotiation.cc:240] Negotiation successful
0806 09:29:34.473169 (+ 580us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":507,"thread_start_us":302,"threads_started":1}
I20260806 09:29:34.475924 1088 heartbeater.cc:347] Connected to a master server at 127.20.113.62:44663
I20260806 09:29:34.476284 1088 heartbeater.cc:464] Registering TS with master...
I20260806 09:29:34.477097 1088 heartbeater.cc:511] Master 127.20.113.62:44663 requested a full tablet report, sending...
I20260806 09:29:34.478749 646 ts_manager.cc:194] Registered new tserver with Master: f88a30a2bfad40dab735bdcad75449c2 (127.20.113.2:40769)
I20260806 09:29:34.480063 646 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.113.2@KRBTEST.COM'} at 127.20.113.2:39279
WARNING: no policy specified for kudu/127.20.113.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.113.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.113.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.3.keytab.
Entry for principal kudu/127.20.113.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.3.keytab.
I20260806 09:29:34.531319 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.113.3:0
--local_ip_for_outbound_sockets=127.20.113.3
--webserver_interface=127.20.113.3
--webserver_port=0
--tserver_master_addrs=127.20.113.62:44663
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.3.keytab
--principal=kudu/127.20.113.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:39947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:29:34.874938 1097 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:29:34.875275 1097 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:29:34.875451 1097 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:29:34.882763 1097 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:29:34.882892 1097 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:29:34.883050 1097 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.113.3
I20260806 09:29:34.895275 1097 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:39947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.3.keytab
--principal=kudu/127.20.113.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.113.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.113.3
--webserver_port=0
--tserver_master_addrs=127.20.113.62:44663
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.113.3
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:29:34.896725 1097 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:29:34.898130 1097 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:29:34.912564 1106 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:29:34.912602 1112 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:29:34.912642 1109 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:29:34.913014 1097 server_base.cc:1034] running on GCE node
I20260806 09:29:34.913930 1097 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:29:34.914870 1097 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:29:34.916109 1097 hybrid_clock.cc:648] HybridClock initialized: now 1786008574916043 us; error 63 us; skew 500 ppm
I20260806 09:29:34.916487 1097 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 09:29:34.918077 903 heartbeater.cc:503] Master 127.20.113.62:44663 was elected leader, sending a full tablet report...
Aug 06 09:29:34 dist-test-slave-45dx krb5kdc[581](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008574, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:29:34.920280 1097 init.cc:377] Logged in from keytab as kudu/127.20.113.3@KRBTEST.COM (short username kudu)
I20260806 09:29:34.921837 1097 webserver.cc:460] Webserver started at http://127.20.113.3:45687/ using document root <none> and password file <none>
I20260806 09:29:34.922569 1097 fs_manager.cc:359] Metadata directory not provided
I20260806 09:29:34.922724 1097 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:29:34.923079 1097 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:29:34.926059 1097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/ts-2/data/instance:
uuid: "b249717f49404135be3b92f9aa5bbec1"
format_stamp: "Formatted at 2026-08-06 09:29:34 on dist-test-slave-45dx"
server_key: "370d08fde65c6a18061b319cee086756"
server_key_iv: "d89f00476b0263d2f59eb84917f053b6"
server_key_version: "encryptionkey@0"
I20260806 09:29:34.926878 1097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/ts-2/wal/instance:
uuid: "b249717f49404135be3b92f9aa5bbec1"
format_stamp: "Formatted at 2026-08-06 09:29:34 on dist-test-slave-45dx"
server_key: "370d08fde65c6a18061b319cee086756"
server_key_iv: "d89f00476b0263d2f59eb84917f053b6"
server_key_version: "encryptionkey@0"
I20260806 09:29:34.932204 1097 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260806 09:29:34.935635 1121 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:29:34.937021 1097 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260806 09:29:34.937217 1097 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/ts-2/wal
uuid: "b249717f49404135be3b92f9aa5bbec1"
format_stamp: "Formatted at 2026-08-06 09:29:34 on dist-test-slave-45dx"
server_key: "370d08fde65c6a18061b319cee086756"
server_key_iv: "d89f00476b0263d2f59eb84917f053b6"
server_key_version: "encryptionkey@0"
I20260806 09:29:34.937430 1097 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:29:34.956781 1097 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:29:34.961874 1097 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:29:34.962208 1097 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:29:34.963244 1097 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:29:34.964774 1097 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:29:34.964884 1097 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:29:34.965055 1097 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:29:34.965138 1097 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:29:34.999878 1097 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.3:45949
I20260806 09:29:34.999979 1287 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.3:45949 every 8 connection(s)
I20260806 09:29:35.001240 1097 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/ts-2/data/info.pb
I20260806 09:29:35.009600 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 1097
I20260806 09:29:35.009828 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/ts-2/wal/instance
I20260806 09:29:35.010375 20932 external_mini_cluster.cc:1428] Setting key 1d2722d7cc7640322c311bb6c4224d7c
Aug 06 09:29:35 dist-test-slave-45dx krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008574, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.3@KRBTEST.COM for kudu/127.20.113.62@KRBTEST.COM
I20260806 09:29:35.022135 1293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:35.004013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.3:50331 (local address 127.20.113.62:44663)
0806 09:29:35.004547 (+ 534us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:35.004552 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:35.005552 (+ 1000us) server_negotiation.cc:407] Connection header received
0806 09:29:35.007399 (+ 1847us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:35.007404 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:35.007481 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:35.007578 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:35.009192 (+ 1614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:35.010164 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:35.011958 (+ 1794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:35.012541 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:35.012669 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:35.016277 (+ 3608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:29:35.016283 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:29:35.016288 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:29:35.016370 (+ 82us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:29:35.019085 (+ 2715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:35.019789 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:35.019795 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:35.019799 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:35.019931 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:35.020392 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:35.020397 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:35.020400 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:35.020664 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:29:35.020840 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:35.021500 (+ 660us) server_negotiation.cc:299] Negotiation successful
0806 09:29:35.021711 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":420,"thread_start_us":246,"threads_started":1}
I20260806 09:29:35.023061 1294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:35.004622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44663 (local address 127.20.113.3:50331)
0806 09:29:35.005361 (+ 739us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:35.005409 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:35.007129 (+ 1720us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:35.007819 (+ 690us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:35.007826 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:35.008444 (+ 618us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:29:35.009030 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:35.009048 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:35.010353 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:35.010357 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:35.011804 (+ 1447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:35.011818 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:35.012770 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:35.012778 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:35.013025 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:35.013748 (+ 723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:29:35.013775 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:29:35.016058 (+ 2283us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:29:35.019230 (+ 3172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:29:35.019238 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:29:35.019253 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:29:35.019661 (+ 408us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:29:35.020048 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:29:35.020053 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:29:35.020056 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:29:35.020248 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:29:35.020809 (+ 561us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:29:35.020817 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:29:35.021132 (+ 315us) client_negotiation.cc:769] Sending connection context
0806 09:29:35.021390 (+ 258us) client_negotiation.cc:240] Negotiation successful
0806 09:29:35.021930 (+ 540us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":275,"threads_started":1}
I20260806 09:29:35.024552 1291 heartbeater.cc:347] Connected to a master server at 127.20.113.62:44663
I20260806 09:29:35.024894 1291 heartbeater.cc:464] Registering TS with master...
I20260806 09:29:35.025902 1291 heartbeater.cc:511] Master 127.20.113.62:44663 requested a full tablet report, sending...
I20260806 09:29:35.027311 646 ts_manager.cc:194] Registered new tserver with Master: b249717f49404135be3b92f9aa5bbec1 (127.20.113.3:45949)
I20260806 09:29:35.028134 646 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.113.3@KRBTEST.COM'} at 127.20.113.3:50331
I20260806 09:29:35.038453 20932 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.20.113.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.113.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.113.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/krb5kdc/oryx_127.20.113.1.keytab.
Entry for principal oryx/127.20.113.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/krb5kdc/oryx_127.20.113.1.keytab.
WARNING: no policy specified for oryx/127.20.113.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.113.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.113.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/krb5kdc/oryx_127.20.113.2.keytab.
Entry for principal oryx/127.20.113.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/krb5kdc/oryx_127.20.113.2.keytab.
WARNING: no policy specified for oryx/127.20.113.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.113.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.113.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/krb5kdc/oryx_127.20.113.3.keytab.
Entry for principal oryx/127.20.113.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/krb5kdc/oryx_127.20.113.3.keytab.
I20260806 09:29:35.228485 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 737
Aug 06 09:29:35 dist-test-slave-45dx krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008572, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.113.1@KRBTEST.COM
I20260806 09:29:35.245410 1304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:35.229668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38748 (local address 127.20.113.1:46421)
0806 09:29:35.230330 (+ 662us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:35.230338 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:35.230365 (+ 27us) server_negotiation.cc:407] Connection header received
0806 09:29:35.230434 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:35.230441 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:35.230645 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:35.230830 (+ 185us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:35.232214 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:35.233279 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:35.234523 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:35.235092 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:35.235204 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:35.239345 (+ 4141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:29:35.239357 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:29:35.239370 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:29:35.239408 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:29:35.242633 (+ 3225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:35.243262 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:35.243270 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:35.243274 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:35.243417 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:35.243757 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:35.243762 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:35.243765 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:35.244161 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:29:35.244480 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:35.244711 (+ 231us) server_negotiation.cc:299] Negotiation successful
0806 09:29:35.244984 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":509,"thread_start_us":338,"threads_started":1}
I20260806 09:29:35.246837 896 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:29:35.327759 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 737
I20260806 09:29:35.356560 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 914
Aug 06 09:29:35 dist-test-slave-45dx krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008572, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.113.2@KRBTEST.COM
I20260806 09:29:35.372543 1306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:35.357550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40294 (local address 127.20.113.2:40769)
0806 09:29:35.358246 (+ 696us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:35.358254 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:35.358280 (+ 26us) server_negotiation.cc:407] Connection header received
0806 09:29:35.358342 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:35.358348 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:35.358553 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:35.358677 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:35.359847 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:35.361061 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:35.362308 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:35.362838 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:35.362940 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:35.367119 (+ 4179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:29:35.367129 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:29:35.367142 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:29:35.367181 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:29:35.369968 (+ 2787us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:35.370565 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:35.370572 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:35.370574 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:35.370694 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:35.371105 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:35.371108 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:35.371110 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:35.371429 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:29:35.371652 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:35.371900 (+ 248us) server_negotiation.cc:299] Negotiation successful
0806 09:29:35.372105 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":552,"thread_start_us":404,"threads_started":1}
I20260806 09:29:35.373930 1083 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:29:35.457355 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 914
I20260806 09:29:35.481256 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 1097
Aug 06 09:29:35 dist-test-slave-45dx krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008572, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.113.3@KRBTEST.COM
I20260806 09:29:35.497967 1308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:35.482321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33408 (local address 127.20.113.3:45949)
0806 09:29:35.482855 (+ 534us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:35.482862 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:35.482904 (+ 42us) server_negotiation.cc:407] Connection header received
0806 09:29:35.482966 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:35.482973 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:35.483187 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:35.483315 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:35.484728 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:35.485727 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:35.487046 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:35.487703 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:35.487800 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:35.492273 (+ 4473us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:29:35.492285 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:29:35.492339 (+ 54us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:29:35.492393 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:29:35.495399 (+ 3006us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:35.496016 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:35.496023 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:35.496025 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:35.496143 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:35.496550 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:35.496553 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:35.496555 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:35.496896 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:29:35.497124 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:35.497347 (+ 223us) server_negotiation.cc:299] Negotiation successful
0806 09:29:35.497552 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":238,"threads_started":1}
I20260806 09:29:35.499265 1282 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:29:35.583709 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 1097
I20260806 09:29:35.611974 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 608
I20260806 09:29:35.613262 687 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:29:35.718837 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 608
I20260806 09:29:35.747524 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.113.62:44663
--webserver_interface=127.20.113.62
--webserver_port=44423
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab
--principal=kudu/127.20.113.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:39947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.113.62:44663
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:29:36.091655 1312 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:29:36.091995 1312 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:29:36.092163 1312 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:29:36.099311 1312 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 09:29:36.099450 1312 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:29:36.099552 1312 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:29:36.099661 1312 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 09:29:36.099730 1312 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 09:29:36.109581 1312 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:39947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.113.62:44663
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab
--principal=kudu/127.20.113.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.113.62:44663
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.113.62
--webserver_port=44423
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:29:36.110955 1312 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:29:36.112417 1312 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:29:36.125026 1322 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:29:36.125710 1319 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:29:36.124997 1320 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:29:36.127121 1312 server_base.cc:1034] running on GCE node
I20260806 09:29:36.127907 1312 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:29:36.129379 1312 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:29:36.130621 1312 hybrid_clock.cc:648] HybridClock initialized: now 1786008576130585 us; error 48 us; skew 500 ppm
I20260806 09:29:36.130986 1312 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:29:36 dist-test-slave-45dx krb5kdc[581](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008576, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:29:36.134073 1312 init.cc:377] Logged in from keytab as kudu/127.20.113.62@KRBTEST.COM (short username kudu)
I20260806 09:29:36.135506 1312 webserver.cc:460] Webserver started at http://127.20.113.62:44423/ using document root <none> and password file <none>
I20260806 09:29:36.136201 1312 fs_manager.cc:359] Metadata directory not provided
I20260806 09:29:36.136438 1312 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:29:36.141609 1312 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.004s
I20260806 09:29:36.144603 1331 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:29:36.145845 1312 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260806 09:29:36.146050 1312 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/master-0/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/master-0/wal
uuid: "35c86c26c9844d93a4b84c16b1682362"
format_stamp: "Formatted at 2026-08-06 09:29:33 on dist-test-slave-45dx"
server_key: "798dbae39d26d73b98b5475cd8b37088"
server_key_iv: "1ddcfdb7ef4743275817f0a2663fb05e"
server_key_version: "encryptionkey@0"
I20260806 09:29:36.147037 1312 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:29:36.166143 1312 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:29:36.170202 1312 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:29:36.170488 1312 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:29:36.189440 1312 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.62:44663
I20260806 09:29:36.189468 1407 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.62:44663 every 8 connection(s)
I20260806 09:29:36.190882 1312 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
I20260806 09:29:36.197695 1408 sys_catalog.cc:263] Verifying existing consensus state
I20260806 09:29:36.198689 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 1312
I20260806 09:29:36.199666 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.113.1:46421
--local_ip_for_outbound_sockets=127.20.113.1
--tserver_master_addrs=127.20.113.62:44663
--webserver_port=36519
--webserver_interface=127.20.113.1
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.1.keytab
--principal=kudu/127.20.113.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:39947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.20.113.1
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/krb5kdc/oryx_127.20.113.1.keytab with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 09:29:36.201094 1408 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 35c86c26c9844d93a4b84c16b1682362: Bootstrap starting.
I20260806 09:29:36.222604 1408 log.cc:824] T 00000000000000000000000000000000 P 35c86c26c9844d93a4b84c16b1682362: Log is configured to *not* fsync() on all Append() calls
I20260806 09:29:36.234901 1408 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 35c86c26c9844d93a4b84c16b1682362: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 09:29:36.235630 1408 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 35c86c26c9844d93a4b84c16b1682362: Bootstrap complete.
I20260806 09:29:36.249605 1408 raft_consensus.cc:348] T 00000000000000000000000000000000 P 35c86c26c9844d93a4b84c16b1682362 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35c86c26c9844d93a4b84c16b1682362" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 44663 } }
I20260806 09:29:36.250869 1408 raft_consensus.cc:729] T 00000000000000000000000000000000 P 35c86c26c9844d93a4b84c16b1682362 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35c86c26c9844d93a4b84c16b1682362, State: Initialized, Role: FOLLOWER
I20260806 09:29:36.251629 1408 consensus_queue.cc:260] T 00000000000000000000000000000000 P 35c86c26c9844d93a4b84c16b1682362 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35c86c26c9844d93a4b84c16b1682362" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 44663 } }
I20260806 09:29:36.251929 1408 raft_consensus.cc:388] T 00000000000000000000000000000000 P 35c86c26c9844d93a4b84c16b1682362 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 09:29:36.252046 1408 raft_consensus.cc:482] T 00000000000000000000000000000000 P 35c86c26c9844d93a4b84c16b1682362 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 09:29:36.252223 1408 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 35c86c26c9844d93a4b84c16b1682362 [term 1 FOLLOWER]: Advancing to term 2
I20260806 09:29:36.257616 1408 raft_consensus.cc:504] T 00000000000000000000000000000000 P 35c86c26c9844d93a4b84c16b1682362 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35c86c26c9844d93a4b84c16b1682362" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 44663 } }
I20260806 09:29:36.258229 1408 leader_election.cc:302] T 00000000000000000000000000000000 P 35c86c26c9844d93a4b84c16b1682362 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 35c86c26c9844d93a4b84c16b1682362; no voters:
I20260806 09:29:36.258874 1408 leader_election.cc:288] T 00000000000000000000000000000000 P 35c86c26c9844d93a4b84c16b1682362 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 09:29:36.259048 1415 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 35c86c26c9844d93a4b84c16b1682362 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 09:29:36.260320 1415 raft_consensus.cc:686] T 00000000000000000000000000000000 P 35c86c26c9844d93a4b84c16b1682362 [term 2 LEADER]: Becoming Leader. State: Replica: 35c86c26c9844d93a4b84c16b1682362, State: Running, Role: LEADER
I20260806 09:29:36.260957 1415 consensus_queue.cc:237] T 00000000000000000000000000000000 P 35c86c26c9844d93a4b84c16b1682362 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35c86c26c9844d93a4b84c16b1682362" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 44663 } }
I20260806 09:29:36.261217 1408 sys_catalog.cc:564] T 00000000000000000000000000000000 P 35c86c26c9844d93a4b84c16b1682362 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 09:29:36.272794 1417 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35c86c26c9844d93a4b84c16b1682362 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "35c86c26c9844d93a4b84c16b1682362" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35c86c26c9844d93a4b84c16b1682362" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 44663 } } }
I20260806 09:29:36.273238 1417 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35c86c26c9844d93a4b84c16b1682362 [sys.catalog]: This master's current role is: LEADER
I20260806 09:29:36.275467 1420 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35c86c26c9844d93a4b84c16b1682362 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 35c86c26c9844d93a4b84c16b1682362. Latest consensus state: current_term: 2 leader_uuid: "35c86c26c9844d93a4b84c16b1682362" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35c86c26c9844d93a4b84c16b1682362" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 44663 } } }
I20260806 09:29:36.275807 1420 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35c86c26c9844d93a4b84c16b1682362 [sys.catalog]: This master's current role is: LEADER
I20260806 09:29:36.277136 1428 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 09:29:36.283938 1428 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 09:29:36.285869 1428 catalog_manager.cc:1259] Loaded cluster ID: 35b03f9873fe493db120b6047fec8f3b
I20260806 09:29:36.286072 1428 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 09:29:36.291101 1428 catalog_manager.cc:1504] Loading token signing keys...
I20260806 09:29:36.293025 1428 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 35c86c26c9844d93a4b84c16b1682362: Loaded TSK: 0
I20260806 09:29:36.295017 1428 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 09:29:36.592351 1412 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:29:36.592648 1412 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:29:36.592803 1412 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:29:36.599854 1412 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:29:36.599952 1412 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:29:36.600116 1412 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.113.1
I20260806 09:29:36.609557 1412 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:39947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/krb5kdc/oryx_127.20.113.1.keytab
--principal=oryx/127.20.113.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.113.1:46421
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.113.1
--webserver_port=36519
--tserver_master_addrs=127.20.113.62:44663
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.113.1
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:29:36.610819 1412 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:29:36.612100 1412 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:29:36.626593 1449 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:29:36.626551 1452 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:29:36.626616 1454 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:29:36.627902 1412 server_base.cc:1034] running on GCE node
I20260806 09:29:36.628623 1412 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:29:36.629650 1412 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:29:36.630882 1412 hybrid_clock.cc:648] HybridClock initialized: now 1786008576630826 us; error 50 us; skew 500 ppm
I20260806 09:29:36.631199 1412 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:29:36 dist-test-slave-45dx krb5kdc[581](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008576, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.113.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:29:36.634541 1412 init.cc:377] Logged in from keytab as oryx/127.20.113.1@KRBTEST.COM (short username oryx)
I20260806 09:29:36.636097 1412 webserver.cc:460] Webserver started at http://127.20.113.1:36519/ using document root <none> and password file <none>
I20260806 09:29:36.636917 1412 fs_manager.cc:359] Metadata directory not provided
I20260806 09:29:36.637091 1412 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:29:36.642674 1412 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 09:29:36.645498 1463 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:29:36.646839 1412 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 09:29:36.647035 1412 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/ts-0/wal
uuid: "63687bd3556c4de6be5505108c3dac7e"
format_stamp: "Formatted at 2026-08-06 09:29:33 on dist-test-slave-45dx"
server_key: "bd56a92f73790bd50124833fb4320936"
server_key_iv: "47297d235f0cb7d920c91f9506b43ec0"
server_key_version: "encryptionkey@0"
I20260806 09:29:36.647984 1412 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:29:36.668128 1412 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:29:36.672055 1412 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:29:36.672421 1412 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:29:36.673398 1412 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:29:36.674767 1412 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:29:36.674903 1412 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:29:36.675052 1412 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:29:36.675143 1412 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:29:36.710019 1412 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.1:46421
I20260806 09:29:36.710125 1618 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.1:46421 every 8 connection(s)
I20260806 09:29:36.711512 1412 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/ts-0/data/info.pb
I20260806 09:29:36.721925 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 1412
I20260806 09:29:36.722743 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.113.2:40769
--local_ip_for_outbound_sockets=127.20.113.2
--tserver_master_addrs=127.20.113.62:44663
--webserver_port=36013
--webserver_interface=127.20.113.2
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.2.keytab
--principal=kudu/127.20.113.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:39947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.20.113.2
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/krb5kdc/oryx_127.20.113.2.keytab with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 09:29:36 dist-test-slave-45dx krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.113.1@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 09:29:36 dist-test-slave-45dx krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.113.1@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM, Server not found in Kerberos database
W20260806 09:29:37.072587 1624 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:29:37.072883 1624 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:29:37.073057 1624 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:29:37.080232 1624 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:29:37.080387 1624 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:29:37.080572 1624 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.113.2
I20260806 09:29:37.089810 1624 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:39947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/krb5kdc/oryx_127.20.113.2.keytab
--principal=oryx/127.20.113.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.113.2:40769
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.113.2
--webserver_port=36013
--tserver_master_addrs=127.20.113.62:44663
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.113.2
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:29:37.091099 1624 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:29:37.092422 1624 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:29:37.110723 1632 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:29:37.110888 1631 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:29:37.111532 1634 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:29:37.112380 1624 server_base.cc:1034] running on GCE node
I20260806 09:29:37.112974 1624 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:29:37.113907 1624 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:29:37.115393 1624 hybrid_clock.cc:648] HybridClock initialized: now 1786008577115318 us; error 76 us; skew 500 ppm
I20260806 09:29:37.115711 1624 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:29:37 dist-test-slave-45dx krb5kdc[581](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008577, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.113.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:29:37.119102 1624 init.cc:377] Logged in from keytab as oryx/127.20.113.2@KRBTEST.COM (short username oryx)
I20260806 09:29:37.120791 1624 webserver.cc:460] Webserver started at http://127.20.113.2:36013/ using document root <none> and password file <none>
I20260806 09:29:37.121469 1624 fs_manager.cc:359] Metadata directory not provided
I20260806 09:29:37.121622 1624 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:29:37.128047 1624 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 09:29:37.132064 1646 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:29:37.133711 1624 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 09:29:37.133900 1624 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/ts-1/wal
uuid: "f88a30a2bfad40dab735bdcad75449c2"
format_stamp: "Formatted at 2026-08-06 09:29:34 on dist-test-slave-45dx"
server_key: "ce4d6ae9c0cd92a155caeadeb0c02bfe"
server_key_iv: "c584945e830e2d8cac72f292d420a1e2"
server_key_version: "encryptionkey@0"
I20260806 09:29:37.134826 1624 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:29:37.151278 1624 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:29:37.156136 1624 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:29:37.156494 1624 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:29:37.157660 1624 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:29:37.159142 1624 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:29:37.159288 1624 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:29:37.159428 1624 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:29:37.159526 1624 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:29:37.206843 1624 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.2:40769
I20260806 09:29:37.206992 1792 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.2:40769 every 8 connection(s)
I20260806 09:29:37.208724 1624 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/ts-1/data/info.pb
I20260806 09:29:37.216825 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 1624
I20260806 09:29:37.217643 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.113.3:45949
--local_ip_for_outbound_sockets=127.20.113.3
--tserver_master_addrs=127.20.113.62:44663
--webserver_port=45687
--webserver_interface=127.20.113.3
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.3.keytab
--principal=kudu/127.20.113.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:39947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.20.113.3
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/krb5kdc/oryx_127.20.113.3.keytab with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 09:29:37 dist-test-slave-45dx krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.113.2@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 09:29:37 dist-test-slave-45dx krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.113.2@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM, Server not found in Kerberos database
W20260806 09:29:37.871744 1800 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:29:37.872107 1800 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:29:37.872344 1800 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:29:37.881289 1800 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:29:37.881402 1800 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:29:37.881639 1800 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.113.3
I20260806 09:29:37.897877 1800 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:39947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/krb5kdc/oryx_127.20.113.3.keytab
--principal=oryx/127.20.113.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.113.3:45949
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.113.3
--webserver_port=45687
--tserver_master_addrs=127.20.113.62:44663
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.113.3
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:29:37.899708 1800 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:29:37.901616 1800 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:29:37.924018 1809 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:29:37.924198 1808 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:29:37.924346 1811 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:29:37.924921 1800 server_base.cc:1034] running on GCE node
I20260806 09:29:37.925590 1800 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:29:37.926784 1800 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:29:37.928017 1800 hybrid_clock.cc:648] HybridClock initialized: now 1786008577927953 us; error 57 us; skew 500 ppm
I20260806 09:29:37.928347 1800 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:29:37 dist-test-slave-45dx krb5kdc[581](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008577, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.113.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:29:37.931515 1800 init.cc:377] Logged in from keytab as oryx/127.20.113.3@KRBTEST.COM (short username oryx)
I20260806 09:29:37.933399 1800 webserver.cc:460] Webserver started at http://127.20.113.3:45687/ using document root <none> and password file <none>
I20260806 09:29:37.934124 1800 fs_manager.cc:359] Metadata directory not provided
I20260806 09:29:37.934293 1800 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:29:37.940021 1800 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.002s
I20260806 09:29:37.944846 1821 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:29:37.946298 1800 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.001s sys 0.003s
I20260806 09:29:37.946478 1800 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/ts-2/wal
uuid: "b249717f49404135be3b92f9aa5bbec1"
format_stamp: "Formatted at 2026-08-06 09:29:34 on dist-test-slave-45dx"
server_key: "370d08fde65c6a18061b319cee086756"
server_key_iv: "d89f00476b0263d2f59eb84917f053b6"
server_key_version: "encryptionkey@0"
I20260806 09:29:37.947438 1800 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:29:37.982931 1800 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:29:37.988240 1800 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:29:37.988570 1800 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:29:37.989888 1800 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:29:37.991585 1800 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:29:37.991737 1800 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:29:37.991870 1800 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:29:37.991950 1800 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
W20260806 09:29:38.026767 1623 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.301s user 0.007s sys 0.011s
I20260806 09:29:38.029383 1623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:36.714784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44663 (local address 127.20.113.1:37199)
0806 09:29:36.715628 (+ 844us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:36.715684 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:36.717505 (+ 1821us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:36.718815 (+ 1310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:36.718822 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:36.719510 (+ 688us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:29:36.720062 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:36.720080 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:36.721721 (+ 1641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:36.721727 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:36.723818 (+ 2091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:36.723835 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:36.724962 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:36.724973 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:36.725316 (+ 343us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:36.726083 (+ 767us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:29:36.726123 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:29:38.027902 (+1301779us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.113.62:44663: Server oryx/127.20.113.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":560,"thread_start_us":380,"threads_started":1}
W20260806 09:29:38.029881 1621 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 09:29:36.714738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.1:37199 (local address 127.20.113.62:44663)
0806 09:29:36.715473 (+ 735us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:36.715485 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:36.715824 (+ 339us) server_negotiation.cc:407] Connection header received
0806 09:29:36.718032 (+ 2208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:36.718046 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:36.718351 (+ 305us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:36.718661 (+ 310us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:36.720240 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:36.721509 (+ 1269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:36.723964 (+ 2455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:36.724712 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:36.724850 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:38.028650 (+1303800us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.113.1:37199: BlockingRecv error: recv got EOF from 127.20.113.1:37199 (error 108)
Metrics: {"server-negotiator.queue_time_us":516,"thread_start_us":304,"threads_started":1}
W20260806 09:29:38.031579 1619 heartbeater.cc:644] Failed to heartbeat to 127.20.113.62:44663 (0 consecutive failures): Not authorized: Failed to ping master at 127.20.113.62:44663: Client connection negotiation failed: client connection to 127.20.113.62:44663: Server oryx/127.20.113.62@KRBTEST.COM not found in Kerberos database
Aug 06 09:29:38 dist-test-slave-45dx krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.113.1@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 09:29:38 dist-test-slave-45dx krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.113.1@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM, Server not found in Kerberos database
W20260806 09:29:38.047851 1621 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 09:29:38.032998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.1:48619 (local address 127.20.113.62:44663)
0806 09:29:38.033211 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:38.033220 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:38.033246 (+ 26us) server_negotiation.cc:407] Connection header received
0806 09:29:38.033337 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:38.033346 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:38.033443 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:38.033568 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:38.034870 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:38.036200 (+ 1330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:38.038624 (+ 2424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:38.039401 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:38.039560 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:38.047474 (+ 7914us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.113.1:48619: BlockingRecv error: recv got EOF from 127.20.113.1:48619 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 09:29:38.048638 1623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:38.032517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44663 (local address 127.20.113.1:48619)
0806 09:29:38.033003 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:38.033036 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:38.033200 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:38.033823 (+ 623us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:38.033829 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:38.034371 (+ 542us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:29:38.034651 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:38.034668 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:38.036401 (+ 1733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:38.036408 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:38.038427 (+ 2019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:38.038444 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:38.039773 (+ 1329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:38.039783 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:38.040104 (+ 321us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:38.041145 (+ 1041us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:29:38.041182 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:29:38.048273 (+ 7091us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.113.62:44663: Server oryx/127.20.113.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":279}
Aug 06 09:29:38 dist-test-slave-45dx krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.113.1@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 09:29:38 dist-test-slave-45dx krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.113.1@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 09:29:38.065726 1623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:38.049491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44663 (local address 127.20.113.1:58345)
0806 09:29:38.049797 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:38.049824 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:38.050090 (+ 266us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:38.051000 (+ 910us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:38.051008 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:38.051736 (+ 728us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:29:38.052370 (+ 634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:38.052389 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:38.054655 (+ 2266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:38.054664 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:38.057112 (+ 2448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:38.057133 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:38.058355 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:38.058365 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:38.058669 (+ 304us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:38.059541 (+ 872us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:29:38.059575 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:29:38.065231 (+ 5656us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.113.62:44663: Server oryx/127.20.113.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":6911,"threads_started":1}
W20260806 09:29:38.065728 1621 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 09:29:38.049631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.1:58345 (local address 127.20.113.62:44663)
0806 09:29:38.049838 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:38.049846 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:38.049932 (+ 86us) server_negotiation.cc:407] Connection header received
0806 09:29:38.050311 (+ 379us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:38.050320 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:38.050446 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:38.050544 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:38.052605 (+ 2061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:38.054325 (+ 1720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:38.057329 (+ 3004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:38.058135 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:38.058282 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:38.065477 (+ 7195us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.113.1:58345: BlockingRecv error: recv got EOF from 127.20.113.1:58345 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
W20260806 09:29:38.066327 1619 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 09:29:38.068078 1800 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.3:45949
I20260806 09:29:38.068146 1982 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.3:45949 every 8 connection(s)
I20260806 09:29:38.069528 1800 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786008495763054-20932-0/minicluster-data/ts-2/data/info.pb
I20260806 09:29:38.075609 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 1800
Aug 06 09:29:38 dist-test-slave-45dx krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.113.3@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 09:29:38 dist-test-slave-45dx krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.113.3@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 09:29:38.104884 1988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:38.077267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50480 (local address 127.20.113.62:44663)
0806 09:29:38.085918 (+ 8651us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:38.085921 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:38.085934 (+ 13us) server_negotiation.cc:407] Connection header received
0806 09:29:38.085969 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:38.085972 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:38.086018 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:38.086135 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:38.087909 (+ 1774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:38.088736 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:38.090463 (+ 1727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:38.090996 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:38.091120 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:38.095789 (+ 4669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:29:38.095803 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:29:38.095827 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:29:38.095881 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:29:38.100381 (+ 4500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:38.101480 (+ 1099us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:38.101492 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:38.101504 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:38.101727 (+ 223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:29:38.102299 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:29:38.102307 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:29:38.102312 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:29:38.102972 (+ 660us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:29:38.103698 (+ 726us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:38.103985 (+ 287us) server_negotiation.cc:299] Negotiation successful
0806 09:29:38.104404 (+ 419us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8567,"thread_start_us":8421,"threads_started":1}
W20260806 09:29:38.437389 1799 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.212s user 0.002s sys 0.018s
W20260806 09:29:38.438920 1797 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 09:29:37.211779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.2:41247 (local address 127.20.113.62:44663)
0806 09:29:37.212351 (+ 572us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:37.212357 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:37.213378 (+ 1021us) server_negotiation.cc:407] Connection header received
0806 09:29:37.215444 (+ 2066us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:37.215454 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:37.215606 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:37.215800 (+ 194us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:37.217773 (+ 1973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:37.219612 (+ 1839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:37.222517 (+ 2905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:37.223322 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:37.223456 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:38.438537 (+1215081us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.113.2:41247: BlockingRecv error: recv got EOF from 127.20.113.2:41247 (error 108)
Metrics: {"server-negotiator.queue_time_us":397,"thread_start_us":272,"threads_started":1}
I20260806 09:29:38.439545 1799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:37.212284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44663 (local address 127.20.113.2:41247)
0806 09:29:37.213181 (+ 897us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:37.213226 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:37.215140 (+ 1914us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:37.215984 (+ 844us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:37.215993 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:37.216697 (+ 704us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:29:37.217533 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:37.217560 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:37.219932 (+ 2372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:37.219942 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:37.222310 (+ 2368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:37.222331 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:37.223883 (+ 1552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:37.223894 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:37.224249 (+ 355us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:37.225263 (+ 1014us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:29:37.225319 (+ 56us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:29:38.438309 (+1212990us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.113.62:44663: Server oryx/127.20.113.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":603,"thread_start_us":353,"threads_started":1}
W20260806 09:29:38.440546 1794 heartbeater.cc:644] Failed to heartbeat to 127.20.113.62:44663 (0 consecutive failures): Not authorized: Failed to ping master at 127.20.113.62:44663: Client connection negotiation failed: client connection to 127.20.113.62:44663: Server oryx/127.20.113.62@KRBTEST.COM not found in Kerberos database
Aug 06 09:29:38 dist-test-slave-45dx krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.113.2@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 09:29:38 dist-test-slave-45dx krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.113.2@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 09:29:38.456149 1799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:38.441331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44663 (local address 127.20.113.2:43733)
0806 09:29:38.441557 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:38.441575 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:38.441705 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:38.442421 (+ 716us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:38.442426 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:38.442822 (+ 396us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:29:38.442994 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:38.443004 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:38.445102 (+ 2098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:38.445106 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:38.446069 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:38.446081 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:38.447372 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:38.447376 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:38.447487 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:38.451586 (+ 4099us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:29:38.451614 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:29:38.455834 (+ 4220us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.113.62:44663: Server oryx/127.20.113.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":90}
W20260806 09:29:38.456269 1797 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 09:29:38.441324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.2:43733 (local address 127.20.113.62:44663)
0806 09:29:38.441488 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:38.441495 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:38.441675 (+ 180us) server_negotiation.cc:407] Connection header received
0806 09:29:38.441881 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:38.441884 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:38.441972 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:38.442076 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:38.443170 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:38.444931 (+ 1761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:38.446224 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:38.447172 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:38.447341 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:38.456020 (+ 8679us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.113.2:43733: BlockingRecv error: recv got EOF from 127.20.113.2:43733 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 06 09:29:38 dist-test-slave-45dx krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.113.2@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 09:29:38 dist-test-slave-45dx krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.113.2@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM, Server not found in Kerberos database
W20260806 09:29:38.466529 1797 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 09:29:38.457486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.2:54455 (local address 127.20.113.62:44663)
0806 09:29:38.457635 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:38.457640 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:38.457750 (+ 110us) server_negotiation.cc:407] Connection header received
0806 09:29:38.457914 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:38.457919 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:38.457985 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:38.458082 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:38.458726 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:38.459443 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:38.460724 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:38.461129 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:38.461261 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:38.466278 (+ 5017us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.113.2:54455: BlockingRecv error: recv got EOF from 127.20.113.2:54455 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 09:29:38.466607 1799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:38.457246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44663 (local address 127.20.113.2:54455)
0806 09:29:38.457665 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:38.457682 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:38.457793 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:38.458120 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:38.458124 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:38.458442 (+ 318us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:29:38.458588 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:38.458596 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:38.459590 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:38.459595 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:38.460588 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:38.460600 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:38.461276 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:38.461282 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:38.461521 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:38.462143 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:29:38.462159 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:29:38.466137 (+ 3978us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.113.62:44663: Server oryx/127.20.113.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":313}
W20260806 09:29:38.467018 1794 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 06 09:29:39 dist-test-slave-45dx krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.113.1@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 09:29:39 dist-test-slave-45dx krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.113.1@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 09:29:39.094662 1994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:39.067114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44663 (local address 127.20.113.1:38941)
0806 09:29:39.081000 (+ 13886us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:39.081032 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:39.081207 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:39.082091 (+ 884us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:39.082100 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:39.082957 (+ 857us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:29:39.083265 (+ 308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:39.083282 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:39.084520 (+ 1238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:39.084529 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:39.086925 (+ 2396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:39.086950 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:39.087792 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:39.087801 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:39.088090 (+ 289us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:39.088943 (+ 853us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:29:39.088979 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:29:39.094407 (+ 5428us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.113.62:44663: Server oryx/127.20.113.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":13680,"thread_start_us":14359,"threads_started":1}
W20260806 09:29:39.095206 1991 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 09:29:39.067122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.1:38941 (local address 127.20.113.62:44663)
0806 09:29:39.081736 (+ 14614us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:39.081740 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:39.081772 (+ 32us) server_negotiation.cc:407] Connection header received
0806 09:29:39.081817 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:39.081820 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:39.081876 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:39.081971 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:39.083542 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:39.084347 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:39.087128 (+ 2781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:39.087540 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:39.087645 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:39.094810 (+ 7165us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.113.1:38941: BlockingRecv error: recv got EOF from 127.20.113.1:38941 (error 108)
Metrics: {"server-negotiator.queue_time_us":14501,"thread_start_us":14384,"threads_started":1}
W20260806 09:29:39.236850 1986 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.150s user 0.005s sys 0.015s
W20260806 09:29:39.238113 1621 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 09:29:38.072226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.3:47393 (local address 127.20.113.62:44663)
0806 09:29:38.072425 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:38.072428 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:38.074302 (+ 1874us) server_negotiation.cc:407] Connection header received
0806 09:29:38.076371 (+ 2069us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:38.076376 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:38.076456 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:38.076592 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:38.079598 (+ 3006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:38.081011 (+ 1413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:38.084334 (+ 3323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:38.085195 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:38.085375 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:39.237840 (+1152465us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.113.3:47393: BlockingRecv error: recv got EOF from 127.20.113.3:47393 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 09:29:39.238806 1986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:38.073251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44663 (local address 127.20.113.3:47393)
0806 09:29:38.074152 (+ 901us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:38.074195 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:38.076066 (+ 1871us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:38.077029 (+ 963us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:38.077046 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:38.078160 (+ 1114us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:29:38.079363 (+ 1203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:38.079393 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:38.081222 (+ 1829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:38.081230 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:38.084108 (+ 2878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:38.084131 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:38.085511 (+ 1380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:38.085522 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:38.085986 (+ 464us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:38.086989 (+ 1003us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:29:38.087034 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:29:39.237661 (+1150627us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.113.62:44663: Server oryx/127.20.113.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":642,"thread_start_us":396,"threads_started":1}
W20260806 09:29:39.239825 1984 heartbeater.cc:644] Failed to heartbeat to 127.20.113.62:44663 (0 consecutive failures): Not authorized: Failed to ping master at 127.20.113.62:44663: Client connection negotiation failed: client connection to 127.20.113.62:44663: Server oryx/127.20.113.62@KRBTEST.COM not found in Kerberos database
Aug 06 09:29:39 dist-test-slave-45dx krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.113.3@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 09:29:39 dist-test-slave-45dx krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.113.3@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 09:29:39.249414 1986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:39.240587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44663 (local address 127.20.113.3:38603)
0806 09:29:39.240820 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:39.240843 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:39.240998 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:39.241384 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:39.241389 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:39.241917 (+ 528us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:29:39.242099 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:39.242108 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:39.243184 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:39.243190 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:39.244170 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:39.244182 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:39.244972 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:39.244978 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:39.245147 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:39.245700 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:29:39.245717 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:29:39.249127 (+ 3410us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.113.62:44663: Server oryx/127.20.113.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":80}
W20260806 09:29:39.249583 1621 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 09:29:39.240801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.3:38603 (local address 127.20.113.62:44663)
0806 09:29:39.241109 (+ 308us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:39.241115 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:39.241132 (+ 17us) server_negotiation.cc:407] Connection header received
0806 09:29:39.241188 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:39.241191 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:39.241247 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:39.241352 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:39.242246 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:39.243026 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:39.244344 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:39.244771 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:39.244922 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:39.249322 (+ 4400us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.113.3:38603: BlockingRecv error: recv got EOF from 127.20.113.3:38603 (error 108)
Metrics: {"server-negotiator.queue_time_us":175}
Aug 06 09:29:39 dist-test-slave-45dx krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.113.3@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 09:29:39 dist-test-slave-45dx krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.113.3@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 09:29:39.258519 1986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:39.250196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44663 (local address 127.20.113.3:49359)
0806 09:29:39.250396 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:39.250417 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:39.250534 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:39.250924 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:39.250928 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:39.251310 (+ 382us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:29:39.251514 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:39.251527 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:39.252779 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:39.252783 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:39.253705 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:39.253716 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:39.254420 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:39.254426 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:39.254610 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:39.255123 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:29:39.255140 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:29:39.258325 (+ 3185us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.113.62:44663: Server oryx/127.20.113.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":70}
W20260806 09:29:39.258723 1621 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 09:29:39.250466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.3:49359 (local address 127.20.113.62:44663)
0806 09:29:39.250651 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:39.250656 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:39.250671 (+ 15us) server_negotiation.cc:407] Connection header received
0806 09:29:39.250730 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:39.250733 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:39.250781 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:39.250897 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:39.251666 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:39.252508 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:39.253846 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:39.254265 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:39.254402 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:39.258519 (+ 4117us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.113.3:49359: BlockingRecv error: recv got EOF from 127.20.113.3:49359 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260806 09:29:39.258898 1984 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 06 09:29:39 dist-test-slave-45dx krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.113.2@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 09:29:39 dist-test-slave-45dx krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.113.2@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 09:29:39.476953 1995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:39.467599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44663 (local address 127.20.113.2:55265)
0806 09:29:39.468138 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:39.468154 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:39.468272 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:39.468634 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:39.468637 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:39.469112 (+ 475us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:29:39.469353 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:39.469370 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:39.470693 (+ 1323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:39.470698 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:39.471706 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:39.471718 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:39.472467 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:39.472472 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:39.472590 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:39.473237 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:29:39.473255 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:29:39.476695 (+ 3440us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.113.62:44663: Server oryx/127.20.113.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":325,"threads_started":1}
W20260806 09:29:39.477147 1621 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 09:29:39.467648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.2:55265 (local address 127.20.113.62:44663)
0806 09:29:39.467878 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:39.467887 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:39.468236 (+ 349us) server_negotiation.cc:407] Connection header received
0806 09:29:39.468413 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:39.468417 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:39.468503 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:39.468587 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:39.469529 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:39.470545 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:39.471831 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:39.472263 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:39.472389 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:39.476907 (+ 4518us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.113.2:55265: BlockingRecv error: recv got EOF from 127.20.113.2:55265 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 06 09:29:40 dist-test-slave-45dx krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.113.1@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 09:29:40 dist-test-slave-45dx krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.113.1@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 09:29:40.105579 1996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:40.096273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44663 (local address 127.20.113.1:47477)
0806 09:29:40.096864 (+ 591us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:40.096881 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:40.097002 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:40.097323 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:40.097326 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:40.097740 (+ 414us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:29:40.097951 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:40.097959 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:40.098935 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:40.098940 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:40.099831 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:40.099842 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:40.100600 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:40.100604 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:40.100736 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:40.101306 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:29:40.101322 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:29:40.105259 (+ 3937us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.113.62:44663: Server oryx/127.20.113.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":269,"threads_started":1}
W20260806 09:29:40.105799 1998 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 09:29:40.096391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.1:47477 (local address 127.20.113.62:44663)
0806 09:29:40.096932 (+ 541us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:40.096936 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:40.096953 (+ 17us) server_negotiation.cc:407] Connection header received
0806 09:29:40.097112 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:40.097115 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:40.097176 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:40.097276 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:40.098088 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:40.098813 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:40.099949 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:40.100462 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:40.100555 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:40.105514 (+ 4959us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.113.1:47477: BlockingRecv error: recv got EOF from 127.20.113.1:47477 (error 108)
Metrics: {"server-negotiator.queue_time_us":445,"thread_start_us":316,"threads_started":1}
Aug 06 09:29:40 dist-test-slave-45dx krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.113.3@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 09:29:40 dist-test-slave-45dx krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.113.3@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 09:29:40.271119 1999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:40.259661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44663 (local address 127.20.113.3:50567)
0806 09:29:40.260248 (+ 587us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:40.260265 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:40.260412 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:40.260767 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:40.260771 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:40.261473 (+ 702us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:29:40.261704 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:40.261720 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:40.263476 (+ 1756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:40.263481 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:40.265351 (+ 1870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:40.265368 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:40.266197 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:40.266205 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:40.266407 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:40.267170 (+ 763us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:29:40.267193 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:29:40.270877 (+ 3684us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.113.62:44663: Server oryx/127.20.113.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":338,"threads_started":1}
W20260806 09:29:40.271322 1998 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 09:29:40.259709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.3:50567 (local address 127.20.113.62:44663)
0806 09:29:40.259894 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:40.259901 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:40.260358 (+ 457us) server_negotiation.cc:407] Connection header received
0806 09:29:40.260533 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:40.260537 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:40.260601 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:40.260686 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:40.261926 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:40.263293 (+ 1367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:40.265557 (+ 2264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:40.266003 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:40.266136 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:40.271078 (+ 4942us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.113.3:50567: BlockingRecv error: recv got EOF from 127.20.113.3:50567 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 06 09:29:40 dist-test-slave-45dx krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.113.2@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 09:29:40 dist-test-slave-45dx krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.113.2@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 09:29:40.486845 2000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:40.477966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44663 (local address 127.20.113.2:55945)
0806 09:29:40.478559 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:40.478576 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:40.478706 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:40.478988 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:40.478991 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:40.479434 (+ 443us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:29:40.479635 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:40.479644 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:40.480674 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:40.480679 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:40.481623 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:40.481638 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:40.482311 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:40.482315 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:40.482442 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:40.483043 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:29:40.483060 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:29:40.486586 (+ 3526us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.113.62:44663: Server oryx/127.20.113.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":351,"threads_started":1}
W20260806 09:29:40.487186 1998 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 09:29:40.478006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.2:55945 (local address 127.20.113.62:44663)
0806 09:29:40.478195 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:40.478199 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:40.478663 (+ 464us) server_negotiation.cc:407] Connection header received
0806 09:29:40.478788 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:40.478792 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:40.478851 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:40.478940 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:40.479759 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:40.480524 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:40.481778 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:40.482176 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:40.482266 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:40.486794 (+ 4528us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.113.2:55945: BlockingRecv error: recv got EOF from 127.20.113.2:55945 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 06 09:29:41 dist-test-slave-45dx krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.113.1@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 09:29:41 dist-test-slave-45dx krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.113.1@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 09:29:41.116181 2001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:41.106680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44663 (local address 127.20.113.1:42817)
0806 09:29:41.107256 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:41.107289 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:41.107399 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:41.107752 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:41.107755 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:41.108133 (+ 378us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:29:41.108413 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:41.108426 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:41.109390 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:41.109393 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:41.110535 (+ 1142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:41.110549 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:41.111347 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:41.111351 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:41.111483 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:41.112064 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:29:41.112081 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:29:41.115925 (+ 3844us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.113.62:44663: Server oryx/127.20.113.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":323,"threads_started":1}
W20260806 09:29:41.116427 2003 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 09:29:41.106853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.1:42817 (local address 127.20.113.62:44663)
0806 09:29:41.107325 (+ 472us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:41.107330 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:41.107347 (+ 17us) server_negotiation.cc:407] Connection header received
0806 09:29:41.107524 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:41.107528 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:41.107589 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:41.107703 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:41.108550 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:41.109268 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:41.110777 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:41.111196 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:41.111294 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:41.116152 (+ 4858us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.113.1:42817: BlockingRecv error: recv got EOF from 127.20.113.1:42817 (error 108)
Metrics: {"server-negotiator.queue_time_us":383,"thread_start_us":274,"threads_started":1}
Aug 06 09:29:41 dist-test-slave-45dx krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.113.3@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 09:29:41 dist-test-slave-45dx krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.113.3@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 09:29:41.284017 2004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:41.272195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44663 (local address 127.20.113.3:59531)
0806 09:29:41.272844 (+ 649us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:41.272866 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:41.273046 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:41.273439 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:41.273445 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:41.274037 (+ 592us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:29:41.274268 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:41.274280 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:41.275855 (+ 1575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:41.275859 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:41.277346 (+ 1487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:41.277363 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:41.278184 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:41.278190 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:41.278388 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:41.279110 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:29:41.279133 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:29:41.283668 (+ 4535us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.113.62:44663: Server oryx/127.20.113.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":496,"thread_start_us":368,"threads_started":1}
W20260806 09:29:41.284246 2003 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 09:29:41.272258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.3:59531 (local address 127.20.113.62:44663)
0806 09:29:41.272509 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:41.272518 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:41.272956 (+ 438us) server_negotiation.cc:407] Connection header received
0806 09:29:41.273170 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:41.273175 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:41.273264 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:41.273358 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:41.274410 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:41.275666 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:41.277522 (+ 1856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:41.278002 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:41.278150 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:41.283958 (+ 5808us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.113.3:59531: BlockingRecv error: recv got EOF from 127.20.113.3:59531 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Aug 06 09:29:41 dist-test-slave-45dx krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.113.2@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 09:29:41 dist-test-slave-45dx krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.113.2@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 09:29:41.496743 2006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:41.487798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44663 (local address 127.20.113.2:47439)
0806 09:29:41.488289 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:41.488326 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:41.488458 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:41.488774 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:41.488777 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:41.489137 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:29:41.489302 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:41.489310 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:41.490283 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:41.490287 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:41.491196 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:41.491207 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:41.491865 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:41.491868 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:41.492001 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:41.492683 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:29:41.492699 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:29:41.496378 (+ 3679us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.113.62:44663: Server oryx/127.20.113.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":278,"threads_started":1}
W20260806 09:29:41.496821 2003 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 09:29:41.487910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.2:47439 (local address 127.20.113.62:44663)
0806 09:29:41.488171 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:41.488177 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:41.488408 (+ 231us) server_negotiation.cc:407] Connection header received
0806 09:29:41.488564 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:41.488568 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:41.488642 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:41.488726 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:41.489424 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:41.490157 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:41.491319 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:41.491741 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:41.491831 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:41.496572 (+ 4741us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.113.2:47439: BlockingRecv error: recv got EOF from 127.20.113.2:47439 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
Aug 06 09:29:42 dist-test-slave-45dx krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.113.1@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 09:29:42 dist-test-slave-45dx krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.113.1@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 09:29:42.126973 2007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:42.117306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44663 (local address 127.20.113.1:57491)
0806 09:29:42.117831 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:42.117847 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:42.117964 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:42.118483 (+ 519us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:42.118486 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:42.118906 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:29:42.119068 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:42.119078 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:42.120110 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:42.120117 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:42.121206 (+ 1089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:42.121220 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:42.121977 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:42.121984 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:42.122177 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:42.122770 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:29:42.122788 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:29:42.126588 (+ 3800us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.113.62:44663: Server oryx/127.20.113.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":292,"threads_started":1}
W20260806 09:29:42.127072 2008 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 09:29:42.117391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.1:57491 (local address 127.20.113.62:44663)
0806 09:29:42.118210 (+ 819us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:42.118215 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:42.118232 (+ 17us) server_negotiation.cc:407] Connection header received
0806 09:29:42.118282 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:42.118286 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:42.118351 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:42.118442 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:42.119234 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:42.119950 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:42.121368 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:42.121815 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:42.121960 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:42.126811 (+ 4851us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.113.1:57491: BlockingRecv error: recv got EOF from 127.20.113.1:57491 (error 108)
Metrics: {"server-negotiator.queue_time_us":731,"thread_start_us":584,"threads_started":1}
Aug 06 09:29:42 dist-test-slave-45dx krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.113.3@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 09:29:42 dist-test-slave-45dx krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.113.3@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 09:29:42.293889 2009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:42.285104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44663 (local address 127.20.113.3:42247)
0806 09:29:42.285679 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:42.285699 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:42.285873 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:42.286293 (+ 420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:42.286301 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:42.286842 (+ 541us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:29:42.287077 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:42.287089 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:42.288176 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:42.288180 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:42.289144 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:42.289156 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:42.289767 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:42.289771 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:42.289898 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:42.290444 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:29:42.290468 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:29:42.293636 (+ 3168us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.113.62:44663: Server oryx/127.20.113.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":298,"threads_started":1}
W20260806 09:29:42.294080 2008 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 09:29:42.285187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.3:42247 (local address 127.20.113.62:44663)
0806 09:29:42.285355 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:42.285359 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:42.285796 (+ 437us) server_negotiation.cc:407] Connection header received
0806 09:29:42.286021 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:42.286025 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:42.286113 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:42.286213 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:42.287300 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:42.288025 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:42.289265 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:42.289650 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:42.289746 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:42.293840 (+ 4094us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.113.3:42247: BlockingRecv error: recv got EOF from 127.20.113.3:42247 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 06 09:29:42 dist-test-slave-45dx krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.113.2@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 09:29:42 dist-test-slave-45dx krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.113.2@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 09:29:42.507089 2012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:42.497873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44663 (local address 127.20.113.2:57801)
0806 09:29:42.498422 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:42.498438 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:42.498602 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:42.498990 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:42.498995 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:42.499412 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:29:42.499579 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:42.499588 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:42.500798 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:42.500804 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:42.501819 (+ 1015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:42.501831 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:42.502550 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:42.502556 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:42.502737 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:42.503342 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:29:42.503359 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:29:42.506824 (+ 3465us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.113.62:44663: Server oryx/127.20.113.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":288,"threads_started":1}
W20260806 09:29:42.507270 2008 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 09:29:42.497955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.2:57801 (local address 127.20.113.62:44663)
0806 09:29:42.498173 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:42.498181 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:42.498537 (+ 356us) server_negotiation.cc:407] Connection header received
0806 09:29:42.498745 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:42.498749 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:42.498823 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:42.498931 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:42.499735 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:42.500661 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:42.501976 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:42.502392 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:42.502537 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:42.507028 (+ 4491us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.113.2:57801: BlockingRecv error: recv got EOF from 127.20.113.2:57801 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 06 09:29:43 dist-test-slave-45dx krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.113.1@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 09:29:43 dist-test-slave-45dx krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.113.1@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 09:29:43.136641 2014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:43.127839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44663 (local address 127.20.113.1:56559)
0806 09:29:43.128409 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:43.128435 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:43.128613 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:43.128966 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:43.128969 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:43.129394 (+ 425us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:29:43.129548 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:43.129557 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:43.130512 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:43.130516 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:43.131404 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:43.131414 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:43.132023 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:43.132027 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:43.132156 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:43.132827 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:29:43.132844 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:29:43.136252 (+ 3408us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.113.62:44663: Server oryx/127.20.113.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":264,"threads_started":1}
W20260806 09:29:43.136845 2015 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 09:29:43.127929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.1:56559 (local address 127.20.113.62:44663)
0806 09:29:43.128536 (+ 607us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:43.128541 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:43.128557 (+ 16us) server_negotiation.cc:407] Connection header received
0806 09:29:43.128755 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:43.128758 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:43.128831 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:43.128920 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:43.129677 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:43.130395 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:43.131522 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:43.131887 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:43.132005 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:43.136566 (+ 4561us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.113.1:56559: BlockingRecv error: recv got EOF from 127.20.113.1:56559 (error 108)
Metrics: {"server-negotiator.queue_time_us":483,"thread_start_us":461,"threads_started":1}
Aug 06 09:29:43 dist-test-slave-45dx krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.113.3@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 09:29:43 dist-test-slave-45dx krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.113.3@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 09:29:43.303689 2016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:43.294829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44663 (local address 127.20.113.3:33695)
0806 09:29:43.295466 (+ 637us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:43.295483 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:43.295599 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:43.295905 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:43.295908 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:43.296292 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:29:43.296513 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:43.296523 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:43.297506 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:43.297510 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:43.298493 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:43.298505 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:43.299117 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:43.299121 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:43.299241 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:43.299838 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:29:43.299855 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:29:43.303309 (+ 3454us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.113.62:44663: Server oryx/127.20.113.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":511,"thread_start_us":369,"threads_started":1}
W20260806 09:29:43.303764 2015 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 09:29:43.294906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.3:33695 (local address 127.20.113.62:44663)
0806 09:29:43.295070 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:43.295074 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:43.295556 (+ 482us) server_negotiation.cc:407] Connection header received
0806 09:29:43.295694 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:43.295698 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:43.295757 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:43.295837 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:43.296639 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:43.297383 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:43.298613 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:43.298997 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:43.299084 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:43.303521 (+ 4437us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.113.3:33695: BlockingRecv error: recv got EOF from 127.20.113.3:33695 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 06 09:29:43 dist-test-slave-45dx krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.113.2@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 09:29:43 dist-test-slave-45dx krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.113.2@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 09:29:43.522094 2020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:43.508359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44663 (local address 127.20.113.2:43679)
0806 09:29:43.509055 (+ 696us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:43.509072 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:43.509264 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:43.509677 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:43.509682 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:43.510265 (+ 583us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:29:43.510425 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:43.510435 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:43.511528 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:43.511535 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:43.512583 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:43.512595 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:43.513390 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:43.513396 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:43.513589 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:43.518177 (+ 4588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:29:43.518194 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:29:43.521728 (+ 3534us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.113.62:44663: Server oryx/127.20.113.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":575,"thread_start_us":377,"threads_started":1}
W20260806 09:29:43.522212 2015 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 09:29:43.508552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.2:43679 (local address 127.20.113.62:44663)
0806 09:29:43.508885 (+ 333us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:43.508892 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:43.509176 (+ 284us) server_negotiation.cc:407] Connection header received
0806 09:29:43.509405 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:43.509411 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:43.509505 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:43.509641 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:43.510582 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:43.511380 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:43.512737 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:43.513231 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:43.513377 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:43.521969 (+ 8592us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.113.2:43679: BlockingRecv error: recv got EOF from 127.20.113.2:43679 (error 108)
Metrics: {"server-negotiator.queue_time_us":103}
Aug 06 09:29:44 dist-test-slave-45dx krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.113.1@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 09:29:44 dist-test-slave-45dx krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.113.1@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 09:29:44.146322 2021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:44.137581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44663 (local address 127.20.113.1:44911)
0806 09:29:44.138208 (+ 627us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:44.138225 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:44.138340 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:44.138647 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:44.138650 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:44.139079 (+ 429us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:29:44.139242 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:44.139250 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:44.140224 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:44.140227 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:44.141149 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:44.141160 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:44.141816 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:44.141820 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:44.141942 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:44.142527 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:29:44.142543 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:29:44.145962 (+ 3419us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.113.62:44663: Server oryx/127.20.113.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":385,"threads_started":1}
W20260806 09:29:44.146435 2023 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 09:29:44.137670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.1:44911 (local address 127.20.113.62:44663)
0806 09:29:44.138188 (+ 518us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:44.138192 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:44.138296 (+ 104us) server_negotiation.cc:407] Connection header received
0806 09:29:44.138441 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:44.138444 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:44.138511 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:44.138598 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:44.139361 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:44.140099 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:44.141267 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:44.141654 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:44.141742 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:44.146155 (+ 4413us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.113.1:44911: BlockingRecv error: recv got EOF from 127.20.113.1:44911 (error 108)
Metrics: {"server-negotiator.queue_time_us":420,"thread_start_us":255,"threads_started":1}
Aug 06 09:29:44 dist-test-slave-45dx krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.113.3@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 09:29:44 dist-test-slave-45dx krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.113.3@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 09:29:44.314661 2024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:44.304713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44663 (local address 127.20.113.3:51355)
0806 09:29:44.305353 (+ 640us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:44.305371 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:44.305539 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:44.305903 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:44.305908 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:44.306371 (+ 463us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:29:44.306523 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:44.306532 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:44.307701 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:44.307708 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:44.308677 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:44.308689 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:44.309420 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:44.309426 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:44.309644 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:44.310350 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:29:44.310368 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:29:44.314346 (+ 3978us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.113.62:44663: Server oryx/127.20.113.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":546,"thread_start_us":332,"threads_started":1}
W20260806 09:29:44.314790 2023 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 09:29:44.304875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.3:51355 (local address 127.20.113.62:44663)
0806 09:29:44.305139 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:44.305146 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:44.305470 (+ 324us) server_negotiation.cc:407] Connection header received
0806 09:29:44.305669 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:44.305674 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:44.305755 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:44.305863 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:44.306678 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:44.307503 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:44.308847 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:44.309252 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:44.309404 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:44.314560 (+ 5156us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.113.3:51355: BlockingRecv error: recv got EOF from 127.20.113.3:51355 (error 108)
Metrics: {"server-negotiator.queue_time_us":98}
Aug 06 09:29:44 dist-test-slave-45dx krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.113.2@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 09:29:44 dist-test-slave-45dx krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.113.2@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 09:29:44.532588 2026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:44.523158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44663 (local address 127.20.113.2:50697)
0806 09:29:44.523658 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:44.523674 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:44.523812 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:44.524120 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:44.524123 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:44.524551 (+ 428us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:29:44.524735 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:44.524744 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:44.525712 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:44.525716 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:44.526645 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:44.526657 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:44.527479 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:44.527483 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:44.527604 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:44.528167 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:29:44.528183 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:29:44.532204 (+ 4021us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.113.62:44663: Server oryx/127.20.113.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":292,"threads_started":1}
W20260806 09:29:44.532716 2023 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 09:29:44.523241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.2:50697 (local address 127.20.113.62:44663)
0806 09:29:44.523497 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:44.523503 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:44.523774 (+ 271us) server_negotiation.cc:407] Connection header received
0806 09:29:44.523908 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:44.523912 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:44.523988 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:44.524069 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:44.524864 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:44.525592 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:44.526792 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:44.527339 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:44.527441 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:44.532461 (+ 5020us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.113.2:50697: BlockingRecv error: recv got EOF from 127.20.113.2:50697 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 06 09:29:45 dist-test-slave-45dx krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.113.1@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 09:29:45 dist-test-slave-45dx krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.113.1@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 09:29:45.158151 2028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:45.147694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44663 (local address 127.20.113.1:56283)
0806 09:29:45.148616 (+ 922us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:45.148633 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:45.148792 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:45.149175 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:45.149179 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:45.149672 (+ 493us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:29:45.149846 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:45.149855 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:45.151137 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:45.151142 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:45.152353 (+ 1211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:45.152365 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:45.153171 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:45.153177 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:45.153361 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:45.154037 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:29:45.154055 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:29:45.157749 (+ 3694us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.113.62:44663: Server oryx/127.20.113.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":791,"thread_start_us":556,"threads_started":1}
W20260806 09:29:45.158255 2027 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 09:29:45.147790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.1:56283 (local address 127.20.113.62:44663)
0806 09:29:45.148353 (+ 563us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:45.148360 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:45.148724 (+ 364us) server_negotiation.cc:407] Connection header received
0806 09:29:45.148938 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:45.148942 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:45.149003 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:45.149120 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:45.150016 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:45.150993 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:45.152493 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:45.152972 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:45.153148 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:45.157975 (+ 4827us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.113.1:56283: BlockingRecv error: recv got EOF from 127.20.113.1:56283 (error 108)
Metrics: {"server-negotiator.queue_time_us":410,"thread_start_us":267,"threads_started":1}
Aug 06 09:29:45 dist-test-slave-45dx krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.113.3@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 09:29:45 dist-test-slave-45dx krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.113.3@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 09:29:45.325492 2030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:45.315815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44663 (local address 127.20.113.3:33621)
0806 09:29:45.316449 (+ 634us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:45.316466 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:45.316630 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:45.317011 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:45.317016 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:45.317698 (+ 682us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:29:45.317962 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:45.317974 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:45.319285 (+ 1311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:45.319290 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:45.320222 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:45.320234 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:45.321002 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:45.321008 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:45.321205 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:45.321767 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:29:45.321786 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:29:45.325238 (+ 3452us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.113.62:44663: Server oryx/127.20.113.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":511,"thread_start_us":313,"threads_started":1}
W20260806 09:29:45.325657 2027 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 09:29:45.315925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.3:33621 (local address 127.20.113.62:44663)
0806 09:29:45.316185 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:45.316193 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:45.316551 (+ 358us) server_negotiation.cc:407] Connection header received
0806 09:29:45.316753 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:45.316759 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:45.316838 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:45.316981 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:45.318106 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:45.319139 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:45.320413 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:45.320837 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:45.320984 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:45.325435 (+ 4451us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.113.3:33621: BlockingRecv error: recv got EOF from 127.20.113.3:33621 (error 108)
Metrics: {"server-negotiator.queue_time_us":97}
Aug 06 09:29:45 dist-test-slave-45dx krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.113.2@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 09:29:45 dist-test-slave-45dx krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.113.2@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 09:29:45.542783 2035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:45.533576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44663 (local address 127.20.113.2:43819)
0806 09:29:45.534247 (+ 671us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:45.534263 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:45.534376 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:45.534683 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:45.534686 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:45.535114 (+ 428us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:29:45.535284 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:45.535294 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:45.536354 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:45.536366 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:45.537504 (+ 1138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:45.537517 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:45.538165 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:45.538169 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:45.538367 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:45.539053 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:29:45.539070 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:29:45.542535 (+ 3465us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.113.62:44663: Server oryx/127.20.113.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":566,"thread_start_us":326,"threads_started":1}
W20260806 09:29:45.542996 2027 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 09:29:45.533627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.2:43819 (local address 127.20.113.62:44663)
0806 09:29:45.533881 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:45.533885 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:45.534338 (+ 453us) server_negotiation.cc:407] Connection header received
0806 09:29:45.534475 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:45.534479 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:45.534550 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:45.534628 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:45.535429 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:45.536174 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:45.537630 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:45.538023 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:45.538110 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:45.542746 (+ 4636us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.113.2:43819: BlockingRecv error: recv got EOF from 127.20.113.2:43819 (error 108)
Metrics: {"server-negotiator.queue_time_us":110}
Aug 06 09:29:46 dist-test-slave-45dx krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.113.1@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 09:29:46 dist-test-slave-45dx krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.113.1@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 09:29:46.169859 2037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:46.159285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44663 (local address 127.20.113.1:57171)
0806 09:29:46.159861 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:46.159883 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:46.160019 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:46.160531 (+ 512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:46.160538 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:46.161105 (+ 567us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:29:46.161343 (+ 238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:46.161360 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:46.162941 (+ 1581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:46.162947 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:46.164358 (+ 1411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:46.164373 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:46.165149 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:46.165155 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:46.165335 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:46.165919 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:29:46.165938 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:29:46.169489 (+ 3551us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.113.62:44663: Server oryx/127.20.113.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":385,"thread_start_us":253,"threads_started":1}
W20260806 09:29:46.169914 2038 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 09:29:46.159359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.1:57171 (local address 127.20.113.62:44663)
0806 09:29:46.160171 (+ 812us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:46.160176 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:46.160193 (+ 17us) server_negotiation.cc:407] Connection header received
0806 09:29:46.160251 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:46.160256 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:46.160363 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:46.160465 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:46.161518 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:46.162788 (+ 1270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:46.164512 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:46.164930 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:46.165076 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:46.169685 (+ 4609us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.113.1:57171: BlockingRecv error: recv got EOF from 127.20.113.1:57171 (error 108)
Metrics: {"server-negotiator.queue_time_us":696,"thread_start_us":601,"threads_started":1}
Aug 06 09:29:46 dist-test-slave-45dx krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.113.3@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 09:29:46 dist-test-slave-45dx krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.113.3@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 09:29:46.336153 2039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:46.326472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44663 (local address 127.20.113.3:35587)
0806 09:29:46.327107 (+ 635us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:46.327124 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:46.327243 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:46.327662 (+ 419us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:46.327666 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:46.328241 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:29:46.328546 (+ 305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:46.328559 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:46.329731 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:46.329735 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:46.330708 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:46.330721 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:46.331481 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:46.331485 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:46.331598 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:46.332185 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:29:46.332202 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:29:46.335904 (+ 3702us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.113.62:44663: Server oryx/127.20.113.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":306,"threads_started":1}
W20260806 09:29:46.336385 2038 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 09:29:46.326650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.3:35587 (local address 127.20.113.62:44663)
0806 09:29:46.326847 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:46.326852 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:46.327214 (+ 362us) server_negotiation.cc:407] Connection header received
0806 09:29:46.327377 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:46.327381 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:46.327466 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:46.327607 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:46.328685 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:46.329594 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:46.330858 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:46.331328 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:46.331441 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:46.336114 (+ 4673us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.113.3:35587: BlockingRecv error: recv got EOF from 127.20.113.3:35587 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 06 09:29:46 dist-test-slave-45dx krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.113.2@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 09:29:46 dist-test-slave-45dx krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.113.2@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 09:29:46.553138 2040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:46.543705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44663 (local address 127.20.113.2:41275)
0806 09:29:46.544235 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:46.544249 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:46.544440 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:46.544774 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:46.544779 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:46.545214 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:29:46.545376 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:46.545386 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:46.546568 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:46.546573 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:46.547497 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:46.547508 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:46.548248 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:46.548254 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:46.548470 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:46.549048 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:29:46.549064 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:29:46.552886 (+ 3822us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.113.62:44663: Server oryx/127.20.113.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":291,"threads_started":1}
W20260806 09:29:46.553292 2038 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 09:29:46.543797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.2:41275 (local address 127.20.113.62:44663)
0806 09:29:46.543988 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:46.543996 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:46.544345 (+ 349us) server_negotiation.cc:407] Connection header received
0806 09:29:46.544561 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:46.544566 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:46.544643 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:46.544740 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:46.545517 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:46.546412 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:46.547646 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:46.548061 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:46.548193 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:46.553082 (+ 4889us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.113.2:41275: BlockingRecv error: recv got EOF from 127.20.113.2:41275 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 06 09:29:47 dist-test-slave-45dx krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.113.1@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 09:29:47 dist-test-slave-45dx krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.113.1@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 09:29:47.179827 2041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:47.170799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44663 (local address 127.20.113.1:36765)
0806 09:29:47.171401 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:47.171417 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:47.171524 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:47.171843 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:47.171846 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:47.172251 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:29:47.172484 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:47.172495 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:47.173463 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:47.173466 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:47.174420 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:47.174431 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:47.175080 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:47.175084 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:47.175203 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:47.175794 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:29:47.175810 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:29:47.179439 (+ 3629us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.113.62:44663: Server oryx/127.20.113.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":336,"threads_started":1}
W20260806 09:29:47.180007 2042 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 09:29:47.170910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.1:36765 (local address 127.20.113.62:44663)
0806 09:29:47.171439 (+ 529us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:47.171443 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:47.171464 (+ 21us) server_negotiation.cc:407] Connection header received
0806 09:29:47.171627 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:47.171630 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:47.171695 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:47.171781 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:47.172609 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:47.173339 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:47.174537 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:47.174953 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:47.175041 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:47.179708 (+ 4667us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.113.1:36765: BlockingRecv error: recv got EOF from 127.20.113.1:36765 (error 108)
Metrics: {"server-negotiator.queue_time_us":420,"thread_start_us":288,"threads_started":1}
Aug 06 09:29:47 dist-test-slave-45dx krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.113.3@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 09:29:47 dist-test-slave-45dx krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.113.3@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 09:29:47.347070 2043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:47.337408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44663 (local address 127.20.113.3:41599)
0806 09:29:47.338064 (+ 656us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:47.338099 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:47.338274 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:47.338627 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:47.338632 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:47.339140 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:29:47.339357 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:47.339368 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:47.340575 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:47.340587 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:47.341544 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:47.341555 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:47.342358 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:47.342365 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:47.342523 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:47.343148 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:29:47.343166 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:29:47.346820 (+ 3654us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.113.62:44663: Server oryx/127.20.113.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":337,"threads_started":1}
W20260806 09:29:47.347501 2042 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 09:29:47.337491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.3:41599 (local address 127.20.113.62:44663)
0806 09:29:47.337710 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:47.337717 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:47.338189 (+ 472us) server_negotiation.cc:407] Connection header received
0806 09:29:47.338406 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:47.338418 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:47.338481 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:47.338589 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:47.339518 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:47.340426 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:47.341691 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:47.342148 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:47.342285 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:47.347065 (+ 4780us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.113.3:41599: BlockingRecv error: recv got EOF from 127.20.113.3:41599 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 06 09:29:47 dist-test-slave-45dx krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.113.2@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 09:29:47 dist-test-slave-45dx krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.113.2@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 09:29:47.563377 2045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:47.553997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44663 (local address 127.20.113.2:56387)
0806 09:29:47.554764 (+ 767us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:47.554779 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:47.554906 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:47.555215 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:47.555218 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:47.555619 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:29:47.555763 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:47.555771 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:47.556786 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:47.556790 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:47.557750 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:47.557761 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:47.558395 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:47.558399 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:47.558527 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:47.559136 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:29:47.559154 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:29:47.563149 (+ 3995us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.113.62:44663: Server oryx/127.20.113.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":667,"thread_start_us":529,"threads_started":1}
W20260806 09:29:47.563598 2042 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 09:29:47.554083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.2:56387 (local address 127.20.113.62:44663)
0806 09:29:47.554326 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:47.554335 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:47.554858 (+ 523us) server_negotiation.cc:407] Connection header received
0806 09:29:47.555013 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:47.555017 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:47.555086 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:47.555165 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:47.555880 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:47.556665 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:47.557870 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:47.558267 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:47.558355 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:47.563338 (+ 4983us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.113.2:56387: BlockingRecv error: recv got EOF from 127.20.113.2:56387 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Aug 06 09:29:48 dist-test-slave-45dx krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.113.1@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 09:29:48 dist-test-slave-45dx krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.113.1@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 09:29:48.189509 2046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:48.181041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44663 (local address 127.20.113.1:33759)
0806 09:29:48.181549 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:48.181565 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:48.181679 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:48.181973 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:48.181976 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:48.182323 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:29:48.182496 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:48.182506 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:48.183437 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:48.183440 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:48.184455 (+ 1015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:48.184467 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:48.185099 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:48.185102 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:48.185205 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:48.185780 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:29:48.185797 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:29:48.189278 (+ 3481us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.113.62:44663: Server oryx/127.20.113.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":268,"threads_started":1}
W20260806 09:29:48.189703 2047 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 09:29:48.181068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.1:33759 (local address 127.20.113.62:44663)
0806 09:29:48.181571 (+ 503us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:48.181576 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:48.181638 (+ 62us) server_negotiation.cc:407] Connection header received
0806 09:29:48.181780 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:48.181784 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:48.181843 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:48.181937 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:48.182617 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:48.183312 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:48.184571 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:48.184975 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:48.185064 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:48.189458 (+ 4394us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.113.1:33759: BlockingRecv error: recv got EOF from 127.20.113.1:33759 (error 108)
Metrics: {"server-negotiator.queue_time_us":405,"thread_start_us":291,"threads_started":1}
Aug 06 09:29:48 dist-test-slave-45dx krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.113.3@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 09:29:48 dist-test-slave-45dx krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.113.3@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 09:29:48.356866 2048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:48.348012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44663 (local address 127.20.113.3:53505)
0806 09:29:48.348571 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:48.348588 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:48.348698 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:48.349004 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:48.349008 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:48.349443 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:29:48.349590 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:48.349598 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:48.350664 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:48.350668 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:48.351823 (+ 1155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:48.351836 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:48.352562 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:48.352567 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:48.352752 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:48.353316 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:29:48.353333 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:29:48.356622 (+ 3289us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.113.62:44663: Server oryx/127.20.113.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":320,"threads_started":1}
W20260806 09:29:48.357038 2047 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 09:29:48.348066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.3:53505 (local address 127.20.113.62:44663)
0806 09:29:48.348236 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:48.348240 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:48.348659 (+ 419us) server_negotiation.cc:407] Connection header received
0806 09:29:48.348794 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:48.348798 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:48.348859 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:48.348947 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:48.349711 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:48.350525 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:48.351964 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:48.352411 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:48.352507 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:48.356797 (+ 4290us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.113.3:53505: BlockingRecv error: recv got EOF from 127.20.113.3:53505 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 06 09:29:48 dist-test-slave-45dx krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.113.2@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 09:29:48 dist-test-slave-45dx krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.113.2@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 09:29:48.577467 2049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:48.564403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44663 (local address 127.20.113.2:44431)
0806 09:29:48.564980 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:48.564998 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:48.565116 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:48.565447 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:48.565451 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:48.565855 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:29:48.566028 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:48.566037 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:48.567035 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:48.567039 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:48.568046 (+ 1007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:48.568058 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:48.568747 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:48.568751 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:48.568876 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:48.573617 (+ 4741us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:29:48.573637 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:29:48.577187 (+ 3550us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.113.62:44663: Server oryx/127.20.113.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":299,"threads_started":1}
W20260806 09:29:48.577682 2047 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 09:29:48.564451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.2:44431 (local address 127.20.113.62:44663)
0806 09:29:48.564615 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:48.564619 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:48.565075 (+ 456us) server_negotiation.cc:407] Connection header received
0806 09:29:48.565237 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:48.565241 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:48.565304 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:48.565386 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:48.566160 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:48.566907 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:48.568182 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:48.568608 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:48.568710 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:48.577347 (+ 8637us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.113.2:44431: BlockingRecv error: recv got EOF from 127.20.113.2:44431 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 06 09:29:49 dist-test-slave-45dx krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.113.1@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 09:29:49 dist-test-slave-45dx krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.113.1@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 09:29:49.201216 2050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:49.190703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44663 (local address 127.20.113.1:44351)
0806 09:29:49.191254 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:49.191277 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:49.191461 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:49.191949 (+ 488us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:49.191954 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:49.192521 (+ 567us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:29:49.192736 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:49.192747 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:49.193823 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:49.193828 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:49.195292 (+ 1464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:49.195307 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:49.196516 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:49.196522 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:49.196704 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:49.197341 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:29:49.197359 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:29:49.201002 (+ 3643us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.113.62:44663: Server oryx/127.20.113.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":427,"thread_start_us":274,"threads_started":1}
W20260806 09:29:49.201476 2052 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 09:29:49.190874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.1:44351 (local address 127.20.113.62:44663)
0806 09:29:49.191675 (+ 801us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:49.191680 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:49.191695 (+ 15us) server_negotiation.cc:407] Connection header received
0806 09:29:49.191738 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:49.191742 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:49.191805 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:49.191892 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:49.192889 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:49.193698 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:49.195602 (+ 1904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:49.196342 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:49.196469 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:49.201201 (+ 4732us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.113.1:44351: BlockingRecv error: recv got EOF from 127.20.113.1:44351 (error 108)
Metrics: {"server-negotiator.queue_time_us":701,"thread_start_us":590,"threads_started":1}
Aug 06 09:29:49 dist-test-slave-45dx krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.113.3@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 09:29:49 dist-test-slave-45dx krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.113.3@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 09:29:49.367177 2053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:49.357781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44663 (local address 127.20.113.3:37511)
0806 09:29:49.358285 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:49.358301 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:49.358427 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:49.358739 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:49.358743 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:49.359139 (+ 396us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:29:49.359300 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:49.359308 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:49.360413 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:49.360417 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:49.361451 (+ 1034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:49.361464 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:49.362162 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:49.362166 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:49.362331 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:49.363094 (+ 763us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:29:49.363135 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:29:49.366908 (+ 3773us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.113.62:44663: Server oryx/127.20.113.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":291,"threads_started":1}
W20260806 09:29:49.367280 2052 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 09:29:49.357857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.3:37511 (local address 127.20.113.62:44663)
0806 09:29:49.358119 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:49.358125 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:49.358386 (+ 261us) server_negotiation.cc:407] Connection header received
0806 09:29:49.358528 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:49.358532 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:49.358596 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:49.358679 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:49.359427 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:49.360219 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:49.361571 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:49.362008 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:49.362100 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:49.367087 (+ 4987us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.113.3:37511: BlockingRecv error: recv got EOF from 127.20.113.3:37511 (error 108)
Metrics: {"server-negotiator.queue_time_us":95}
Aug 06 09:29:49 dist-test-slave-45dx krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.113.2@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 09:29:49 dist-test-slave-45dx krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.113.2@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 09:29:49.588076 2054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:49.578525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44663 (local address 127.20.113.2:38371)
0806 09:29:49.579297 (+ 772us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:49.579313 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:49.579472 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:49.579918 (+ 446us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:49.579923 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:49.580486 (+ 563us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:29:49.580680 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:49.580690 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:49.581763 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:49.581768 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:49.582712 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:49.582725 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:49.583412 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:49.583416 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:49.583550 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:49.584205 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:29:49.584222 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:29:49.587842 (+ 3620us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.113.62:44663: Server oryx/127.20.113.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":661,"thread_start_us":543,"threads_started":1}
W20260806 09:29:49.588317 2052 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 09:29:49.578625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.2:38371 (local address 127.20.113.62:44663)
0806 09:29:49.578859 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:49.578866 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:49.579412 (+ 546us) server_negotiation.cc:407] Connection header received
0806 09:29:49.579623 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:49.579628 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:49.579743 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:49.579867 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:49.580818 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:49.581597 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:49.582856 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:49.583271 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:49.583369 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:49.588045 (+ 4676us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.113.2:38371: BlockingRecv error: recv got EOF from 127.20.113.2:38371 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 06 09:29:50 dist-test-slave-45dx krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.113.1@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 09:29:50 dist-test-slave-45dx krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.113.1@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 09:29:50.211526 2056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:50.202183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44663 (local address 127.20.113.1:40803)
0806 09:29:50.202733 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:50.202751 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:50.202883 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:50.203198 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:50.203202 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:50.203606 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:29:50.203780 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:50.203788 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:50.204850 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:50.204854 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:50.205779 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:50.205790 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:50.206498 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:50.206502 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:50.206633 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:50.207333 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:29:50.207351 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:29:50.211183 (+ 3832us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.113.62:44663: Server oryx/127.20.113.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":285,"threads_started":1}
W20260806 09:29:50.211817 2057 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 09:29:50.202250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.1:40803 (local address 127.20.113.62:44663)
0806 09:29:50.202758 (+ 508us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:50.202763 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:50.202829 (+ 66us) server_negotiation.cc:407] Connection header received
0806 09:29:50.202993 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:50.202997 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:50.203065 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:50.203155 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:50.203909 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:50.204722 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:50.205900 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:50.206334 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:50.206434 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:50.211496 (+ 5062us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.113.1:40803: BlockingRecv error: recv got EOF from 127.20.113.1:40803 (error 108)
Metrics: {"server-negotiator.queue_time_us":415,"thread_start_us":284,"threads_started":1}
Aug 06 09:29:50 dist-test-slave-45dx krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.113.3@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 09:29:50 dist-test-slave-45dx krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.113.3@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 09:29:50.377739 2058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:50.368202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44663 (local address 127.20.113.3:52921)
0806 09:29:50.368816 (+ 614us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:50.368833 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:50.368953 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:50.369249 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:50.369252 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:50.369686 (+ 434us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:29:50.369876 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:50.369885 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:50.370890 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:50.370893 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:50.371832 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:50.371844 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:50.372539 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:50.372544 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:50.372686 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:50.373409 (+ 723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:29:50.373429 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:29:50.377330 (+ 3901us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.113.62:44663: Server oryx/127.20.113.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":361,"threads_started":1}
W20260806 09:29:50.377933 2057 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 09:29:50.368243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.3:52921 (local address 127.20.113.62:44663)
0806 09:29:50.368438 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:50.368443 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:50.368912 (+ 469us) server_negotiation.cc:407] Connection header received
0806 09:29:50.369050 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:50.369053 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:50.369116 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:50.369199 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:50.370006 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:50.370767 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:50.371958 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:50.372409 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:50.372504 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:50.377674 (+ 5170us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.113.3:52921: BlockingRecv error: recv got EOF from 127.20.113.3:52921 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Aug 06 09:29:50 dist-test-slave-45dx krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.113.2@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 09:29:50 dist-test-slave-45dx krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.113.2@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 09:29:50.598762 2059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:50.589198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44663 (local address 127.20.113.2:43261)
0806 09:29:50.589811 (+ 613us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:50.589827 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:50.589944 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:50.590274 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:50.590277 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:50.590711 (+ 434us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:29:50.590894 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:50.590903 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:50.591913 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:50.591918 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:50.593000 (+ 1082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:50.593014 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:50.593937 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:50.593942 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:50.594072 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:50.594694 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:29:50.594712 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:29:50.598391 (+ 3679us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.113.62:44663: Server oryx/127.20.113.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":325,"threads_started":1}
W20260806 09:29:50.599138 2057 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 09:29:50.589251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.2:43261 (local address 127.20.113.62:44663)
0806 09:29:50.589451 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:50.589455 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:50.589904 (+ 449us) server_negotiation.cc:407] Connection header received
0806 09:29:50.590044 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:50.590048 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:50.590125 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:50.590208 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:50.591023 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:50.591750 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:50.593136 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:50.593766 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:50.593864 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:50.598602 (+ 4738us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.113.2:43261: BlockingRecv error: recv got EOF from 127.20.113.2:43261 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
Aug 06 09:29:51 dist-test-slave-45dx krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.113.1@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 09:29:51 dist-test-slave-45dx krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.113.1@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 09:29:51.221776 2060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:51.212508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44663 (local address 127.20.113.1:41333)
0806 09:29:51.213006 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:51.213031 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:51.213133 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:51.213855 (+ 722us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:51.213867 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:51.214399 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:29:51.214574 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:51.214583 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:51.215612 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:51.215616 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:51.216524 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:51.216555 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:51.217272 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:51.217276 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:51.217400 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:51.218078 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:29:51.218096 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:29:51.221502 (+ 3406us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.113.62:44663: Server oryx/127.20.113.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":274,"threads_started":1}
W20260806 09:29:51.222051 2062 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 09:29:51.212988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.1:41333 (local address 127.20.113.62:44663)
0806 09:29:51.213491 (+ 503us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:51.213495 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:51.213536 (+ 41us) server_negotiation.cc:407] Connection header received
0806 09:29:51.213593 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:51.213598 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:51.213673 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:51.213788 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:51.214724 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:51.215485 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:51.216681 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:51.217143 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:51.217240 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:51.221732 (+ 4492us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.113.1:41333: BlockingRecv error: recv got EOF from 127.20.113.1:41333 (error 108)
Metrics: {"server-negotiator.queue_time_us":416,"thread_start_us":254,"threads_started":1}
Aug 06 09:29:51 dist-test-slave-45dx krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.113.3@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 09:29:51 dist-test-slave-45dx krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.113.3@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 09:29:51.388289 2063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:51.378907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44663 (local address 127.20.113.3:54137)
0806 09:29:51.379499 (+ 592us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:51.379515 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:51.379662 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:51.380086 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:51.380091 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:51.380740 (+ 649us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:29:51.380916 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:51.380927 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:51.382042 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:51.382047 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:51.382982 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:51.382994 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:51.383723 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:51.383729 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:51.383919 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:51.384547 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:29:51.384565 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:29:51.388048 (+ 3483us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.113.62:44663: Server oryx/127.20.113.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":312,"threads_started":1}
W20260806 09:29:51.388499 2062 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 09:29:51.378997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.3:54137 (local address 127.20.113.62:44663)
0806 09:29:51.379175 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:51.379179 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:51.379610 (+ 431us) server_negotiation.cc:407] Connection header received
0806 09:29:51.379833 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:51.379838 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:51.379925 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:51.380038 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:51.381076 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:51.381885 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:51.383158 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:51.383575 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:51.383707 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:51.388227 (+ 4520us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.113.3:54137: BlockingRecv error: recv got EOF from 127.20.113.3:54137 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 06 09:29:51 dist-test-slave-45dx krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.113.2@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 09:29:51 dist-test-slave-45dx krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.113.2@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 09:29:51.608853 2064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:51.599806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44663 (local address 127.20.113.2:53213)
0806 09:29:51.600372 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:51.600401 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:51.600570 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:51.600935 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:51.600939 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:51.601393 (+ 454us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:29:51.601546 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:51.601555 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:51.602609 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:51.602615 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:51.603537 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:51.603548 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:51.604243 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:51.604249 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:51.604451 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:51.605151 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:29:51.605170 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:29:51.608617 (+ 3447us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.113.62:44663: Server oryx/127.20.113.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":429,"thread_start_us":311,"threads_started":1}
W20260806 09:29:51.609203 2062 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 09:29:51.600034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.2:53213 (local address 127.20.113.62:44663)
0806 09:29:51.600270 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:51.600277 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:51.600498 (+ 221us) server_negotiation.cc:407] Connection header received
0806 09:29:51.600692 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:51.600697 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:51.600785 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:51.600907 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:51.601690 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:51.602459 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:51.603679 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:51.604098 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:51.604229 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:51.608805 (+ 4576us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.113.2:53213: BlockingRecv error: recv got EOF from 127.20.113.2:53213 (error 108)
Metrics: {"server-negotiator.queue_time_us":97}
Aug 06 09:29:52 dist-test-slave-45dx krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.113.1@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 09:29:52 dist-test-slave-45dx krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.113.1@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 09:29:52.232682 2066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:52.223058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44663 (local address 127.20.113.1:58619)
0806 09:29:52.223602 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:52.223619 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:52.223718 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:52.224032 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:52.224035 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:52.224443 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:29:52.224623 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:52.224632 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:52.225657 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:52.225661 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:52.226835 (+ 1174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:52.226853 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:52.227915 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:52.227919 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:52.228055 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:52.228726 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:29:52.228744 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:29:52.232400 (+ 3656us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.113.62:44663: Server oryx/127.20.113.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":286,"threads_started":1}
W20260806 09:29:52.232890 2067 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 09:29:52.223196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.1:58619 (local address 127.20.113.62:44663)
0806 09:29:52.223757 (+ 561us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:52.223761 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:52.223778 (+ 17us) server_negotiation.cc:407] Connection header received
0806 09:29:52.223831 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:52.223835 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:52.223892 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:52.223983 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:52.224776 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:52.225530 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:52.227030 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:52.227764 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:52.227914 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:52.232621 (+ 4707us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.113.1:58619: BlockingRecv error: recv got EOF from 127.20.113.1:58619 (error 108)
Metrics: {"server-negotiator.queue_time_us":470,"thread_start_us":348,"threads_started":1}
Aug 06 09:29:52 dist-test-slave-45dx krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.113.3@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 09:29:52 dist-test-slave-45dx krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.113.3@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 09:29:52.398772 2068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:52.389568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44663 (local address 127.20.113.3:50609)
0806 09:29:52.390075 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:52.390091 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:52.390213 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:52.390504 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:52.390508 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:52.390935 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:29:52.391084 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:52.391092 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:52.392147 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:52.392151 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:52.393033 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:52.393044 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:52.393815 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:52.393821 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:52.394008 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:52.394717 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:29:52.394740 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:29:52.398531 (+ 3791us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.113.62:44663: Server oryx/127.20.113.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":283,"threads_started":1}
W20260806 09:29:52.398995 2067 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 09:29:52.389664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.3:50609 (local address 127.20.113.62:44663)
0806 09:29:52.390003 (+ 339us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:52.390009 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:52.390202 (+ 193us) server_negotiation.cc:407] Connection header received
0806 09:29:52.390302 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:52.390306 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:52.390371 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:52.390458 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:52.391228 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:52.392004 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:52.393184 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:52.393636 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:52.393779 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:52.398742 (+ 4963us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.113.3:50609: BlockingRecv error: recv got EOF from 127.20.113.3:50609 (error 108)
Metrics: {"server-negotiator.queue_time_us":201}
Aug 06 09:29:52 dist-test-slave-45dx krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.113.2@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 09:29:52 dist-test-slave-45dx krb5kdc[581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.113.2@KRBTEST.COM for oryx/127.20.113.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 09:29:52.619593 2069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:52.610040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44663 (local address 127.20.113.2:36845)
0806 09:29:52.610593 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:52.610609 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:52.610784 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:52.611155 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:52.611160 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:52.611585 (+ 425us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:29:52.611785 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:52.611795 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:52.612934 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:52.612940 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:52.613877 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:52.613889 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:52.614618 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:52.614625 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:52.614815 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:52.615450 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:29:52.615467 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:29:52.619190 (+ 3723us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.113.62:44663: Server oryx/127.20.113.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":287,"threads_started":1}
W20260806 09:29:52.619695 2067 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 09:29:52.610149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.2:36845 (local address 127.20.113.62:44663)
0806 09:29:52.610393 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:52.610402 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:52.610696 (+ 294us) server_negotiation.cc:407] Connection header received
0806 09:29:52.610922 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:52.610931 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:52.611008 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:52.611123 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:52.611940 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:52.612778 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:52.614028 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:52.614459 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:52.614603 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:52.619408 (+ 4805us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.113.2:36845: BlockingRecv error: recv got EOF from 127.20.113.2:36845 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
I20260806 09:29:53.080174 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 1412
I20260806 09:29:53.092626 2071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:53.081347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42946 (local address 127.20.113.1:46421)
0806 09:29:53.082008 (+ 661us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:53.082016 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:53.082043 (+ 27us) server_negotiation.cc:407] Connection header received
0806 09:29:53.082118 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:53.082125 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:53.082358 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:53.082607 (+ 249us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:53.083918 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:53.084981 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:53.086138 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:53.086650 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:53.086772 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:53.089300 (+ 2528us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:29:53.089308 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:29:53.089321 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:29:53.089379 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:29:53.092076 (+ 2697us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.20.113.1@KRBTEST.COM not found in keytab (ticket kvno 2)
0806 09:29:53.092413 (+ 337us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:42946: Request ticket server kudu/127.20.113.1@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":466,"thread_start_us":371,"threads_started":1}
W20260806 09:29:53.092754 2071 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:42946: Request ticket server kudu/127.20.113.1@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260806 09:29:53.092772 20932 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 1412: Not authorized: Client connection negotiation failed: client connection to 127.20.113.1:46421: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.20.113.1@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260806 09:29:53.093019 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 1412
I20260806 09:29:53.120380 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 1624
I20260806 09:29:53.133373 2072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:53.121447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49526 (local address 127.20.113.2:40769)
0806 09:29:53.122093 (+ 646us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:53.122101 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:53.122127 (+ 26us) server_negotiation.cc:407] Connection header received
0806 09:29:53.122192 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:53.122198 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:53.122428 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:53.122577 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:53.124517 (+ 1940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:53.125576 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:53.126807 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:53.127350 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:53.127449 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:53.129947 (+ 2498us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:29:53.129955 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:29:53.129969 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:29:53.130008 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:29:53.132820 (+ 2812us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.20.113.2@KRBTEST.COM not found in keytab (ticket kvno 2)
0806 09:29:53.133106 (+ 286us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:49526: Request ticket server kudu/127.20.113.2@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":481,"thread_start_us":345,"threads_started":1}
W20260806 09:29:53.133479 2072 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:49526: Request ticket server kudu/127.20.113.2@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260806 09:29:53.133493 20932 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 1624: Not authorized: Client connection negotiation failed: client connection to 127.20.113.2:40769: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.20.113.2@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260806 09:29:53.133649 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 1624
I20260806 09:29:53.160529 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 1800
I20260806 09:29:53.173664 2074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:53.161799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50702 (local address 127.20.113.3:45949)
0806 09:29:53.162467 (+ 668us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:53.162474 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:53.162499 (+ 25us) server_negotiation.cc:407] Connection header received
0806 09:29:53.162562 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:53.162569 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:53.162909 (+ 340us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:53.163167 (+ 258us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:53.164669 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:53.165696 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:53.166884 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:53.167466 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:53.167622 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:53.170356 (+ 2734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:29:53.170365 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:29:53.170378 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:29:53.170426 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:29:53.172950 (+ 2524us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.20.113.3@KRBTEST.COM not found in keytab (ticket kvno 2)
0806 09:29:53.173323 (+ 373us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:50702: Request ticket server kudu/127.20.113.3@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":485,"thread_start_us":343,"threads_started":1}
W20260806 09:29:53.173704 20932 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 1800: Not authorized: Client connection negotiation failed: client connection to 127.20.113.3:45949: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.20.113.3@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260806 09:29:53.173786 2074 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:50702: Request ticket server kudu/127.20.113.3@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260806 09:29:53.173895 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 1800
I20260806 09:29:53.199524 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 1312
I20260806 09:29:53.201083 1399 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:29:53.321273 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 1312
2026-08-06T09:29:53Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (22100 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-06T09:29:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T09:29:53Z Disabled control of system clock
I20260806 09:29:53.425041 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786008495763054-20932-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786008495763054-20932-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786008495763054-20932-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.113.62:45327
--webserver_interface=127.20.113.62
--webserver_port=0
--builtin_ntp_servers=127.20.113.20:35497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.113.62:45327
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260806 09:29:53.772277 2083 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:29:53.772687 2083 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:29:53.772825 2083 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:29:53.780081 2083 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 09:29:53.780226 2083 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:29:53.780316 2083 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:29:53.780435 2083 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 09:29:53.780514 2083 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 09:29:53.789914 2083 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:35497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786008495763054-20932-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786008495763054-20932-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.113.62:45327
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.113.62:45327
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.113.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786008495763054-20932-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:29:53.791324 2083 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:29:53.793354 2083 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:29:53.805349 2097 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:29:53.805533 2092 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:29:53.805670 2091 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:29:53.807102 2083 server_base.cc:1034] running on GCE node
I20260806 09:29:53.807848 2083 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:29:53.809309 2083 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:29:53.810567 2083 hybrid_clock.cc:648] HybridClock initialized: now 1786008593810530 us; error 53 us; skew 500 ppm
I20260806 09:29:53.810943 2083 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 09:29:53.818404 2083 webserver.cc:460] Webserver started at http://127.20.113.62:44199/ using document root <none> and password file <none>
I20260806 09:29:53.819159 2083 fs_manager.cc:359] Metadata directory not provided
I20260806 09:29:53.819301 2083 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:29:53.819653 2083 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:29:53.822926 2083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786008495763054-20932-0/minicluster-data/master-0/data/instance:
uuid: "b87318cc994344f1ae124f1ae3873c18"
format_stamp: "Formatted at 2026-08-06 09:29:53 on dist-test-slave-45dx"
server_key: "b9d3045db9011178fd787a498722a638"
server_key_iv: "aa8982d1018e4dd589724f598600c580"
server_key_version: "encryptionkey@0"
I20260806 09:29:53.823747 2083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786008495763054-20932-0/minicluster-data/master-0/wal/instance:
uuid: "b87318cc994344f1ae124f1ae3873c18"
format_stamp: "Formatted at 2026-08-06 09:29:53 on dist-test-slave-45dx"
server_key: "b9d3045db9011178fd787a498722a638"
server_key_iv: "aa8982d1018e4dd589724f598600c580"
server_key_version: "encryptionkey@0"
I20260806 09:29:53.828974 2083 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 09:29:53.832386 2104 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:29:53.833762 2083 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 09:29:53.833964 2083 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786008495763054-20932-0/minicluster-data/master-0/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786008495763054-20932-0/minicluster-data/master-0/wal
uuid: "b87318cc994344f1ae124f1ae3873c18"
format_stamp: "Formatted at 2026-08-06 09:29:53 on dist-test-slave-45dx"
server_key: "b9d3045db9011178fd787a498722a638"
server_key_iv: "aa8982d1018e4dd589724f598600c580"
server_key_version: "encryptionkey@0"
I20260806 09:29:53.834188 2083 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786008495763054-20932-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786008495763054-20932-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786008495763054-20932-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:29:53.859608 2083 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:29:53.860565 2083 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:29:53.860867 2083 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:29:53.879827 2083 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.62:45327
I20260806 09:29:53.879853 2173 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.62:45327 every 8 connection(s)
I20260806 09:29:53.881314 2083 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
I20260806 09:29:53.884438 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 2083
I20260806 09:29:53.884652 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786008495763054-20932-0/minicluster-data/master-0/wal/instance
I20260806 09:29:53.885326 20932 external_mini_cluster.cc:1428] Setting key 93f92e77932b3b52d7525063ad088c12
I20260806 09:29:53.885473 2175 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:29:53.899283 2175 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b87318cc994344f1ae124f1ae3873c18: Bootstrap starting.
I20260806 09:29:53.899284 2182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:53.887521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55278 (local address 127.20.113.62:45327)
0806 09:29:53.888646 (+ 1125us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:53.888657 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:53.888702 (+ 45us) server_negotiation.cc:407] Connection header received
0806 09:29:53.890203 (+ 1501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:53.890224 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:53.890676 (+ 452us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:53.890977 (+ 301us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:53.891682 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:53.893047 (+ 1365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:53.894807 (+ 1760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:53.895478 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:53.895608 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:53.897001 (+ 1393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:29:53.897008 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:29:53.897020 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:29:53.897571 (+ 551us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:29:53.897649 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:29:53.897670 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:29:53.897753 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:53.898015 (+ 262us) server_negotiation.cc:299] Negotiation successful
0806 09:29:53.898355 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":610,"thread_start_us":398,"threads_started":1}
I20260806 09:29:53.902642 2175 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b87318cc994344f1ae124f1ae3873c18: Neither blocks nor log segments found. Creating new log.
I20260806 09:29:53.903693 2175 log.cc:824] T 00000000000000000000000000000000 P b87318cc994344f1ae124f1ae3873c18: Log is configured to *not* fsync() on all Append() calls
I20260806 09:29:53.906951 2175 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b87318cc994344f1ae124f1ae3873c18: No bootstrap required, opened a new log
I20260806 09:29:53.915234 2175 raft_consensus.cc:348] T 00000000000000000000000000000000 P b87318cc994344f1ae124f1ae3873c18 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b87318cc994344f1ae124f1ae3873c18" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 45327 } }
I20260806 09:29:53.915647 2175 raft_consensus.cc:374] T 00000000000000000000000000000000 P b87318cc994344f1ae124f1ae3873c18 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:29:53.915752 2175 raft_consensus.cc:729] T 00000000000000000000000000000000 P b87318cc994344f1ae124f1ae3873c18 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b87318cc994344f1ae124f1ae3873c18, State: Initialized, Role: FOLLOWER
I20260806 09:29:53.916508 2175 consensus_queue.cc:260] T 00000000000000000000000000000000 P b87318cc994344f1ae124f1ae3873c18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b87318cc994344f1ae124f1ae3873c18" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 45327 } }
I20260806 09:29:53.916733 2175 raft_consensus.cc:388] T 00000000000000000000000000000000 P b87318cc994344f1ae124f1ae3873c18 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 09:29:53.916914 2175 raft_consensus.cc:482] T 00000000000000000000000000000000 P b87318cc994344f1ae124f1ae3873c18 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 09:29:53.917085 2175 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b87318cc994344f1ae124f1ae3873c18 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:29:53.919307 2175 raft_consensus.cc:504] T 00000000000000000000000000000000 P b87318cc994344f1ae124f1ae3873c18 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b87318cc994344f1ae124f1ae3873c18" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 45327 } }
I20260806 09:29:53.919834 2175 leader_election.cc:302] T 00000000000000000000000000000000 P b87318cc994344f1ae124f1ae3873c18 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b87318cc994344f1ae124f1ae3873c18; no voters:
I20260806 09:29:53.920523 2175 leader_election.cc:288] T 00000000000000000000000000000000 P b87318cc994344f1ae124f1ae3873c18 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 09:29:53.920732 2185 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b87318cc994344f1ae124f1ae3873c18 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:29:53.921938 2185 raft_consensus.cc:686] T 00000000000000000000000000000000 P b87318cc994344f1ae124f1ae3873c18 [term 1 LEADER]: Becoming Leader. State: Replica: b87318cc994344f1ae124f1ae3873c18, State: Running, Role: LEADER
I20260806 09:29:53.922533 2185 consensus_queue.cc:237] T 00000000000000000000000000000000 P b87318cc994344f1ae124f1ae3873c18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b87318cc994344f1ae124f1ae3873c18" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 45327 } }
I20260806 09:29:53.922797 2175 sys_catalog.cc:564] T 00000000000000000000000000000000 P b87318cc994344f1ae124f1ae3873c18 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 09:29:53.925810 2189 sys_catalog.cc:455] T 00000000000000000000000000000000 P b87318cc994344f1ae124f1ae3873c18 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b87318cc994344f1ae124f1ae3873c18. Latest consensus state: current_term: 1 leader_uuid: "b87318cc994344f1ae124f1ae3873c18" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b87318cc994344f1ae124f1ae3873c18" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 45327 } } }
I20260806 09:29:53.926208 2189 sys_catalog.cc:458] T 00000000000000000000000000000000 P b87318cc994344f1ae124f1ae3873c18 [sys.catalog]: This master's current role is: LEADER
I20260806 09:29:53.925911 2186 sys_catalog.cc:455] T 00000000000000000000000000000000 P b87318cc994344f1ae124f1ae3873c18 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b87318cc994344f1ae124f1ae3873c18" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b87318cc994344f1ae124f1ae3873c18" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 45327 } } }
I20260806 09:29:53.926335 2186 sys_catalog.cc:458] T 00000000000000000000000000000000 P b87318cc994344f1ae124f1ae3873c18 [sys.catalog]: This master's current role is: LEADER
I20260806 09:29:53.928376 2195 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 09:29:53.934194 2195 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 09:29:53.944141 2195 catalog_manager.cc:1347] Generated new cluster ID: 3d3b403584354044babb8b0f23a9e748
I20260806 09:29:53.944293 2195 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 09:29:53.957751 2195 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 09:29:53.959142 2195 catalog_manager.cc:1504] Loading token signing keys...
I20260806 09:29:53.974218 2195 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b87318cc994344f1ae124f1ae3873c18: Generated new TSK 0
I20260806 09:29:53.975085 2195 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 09:29:53.985605 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786008495763054-20932-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786008495763054-20932-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786008495763054-20932-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786008495763054-20932-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.113.1:0
--local_ip_for_outbound_sockets=127.20.113.1
--webserver_interface=127.20.113.1
--webserver_port=0
--tserver_master_addrs=127.20.113.62:45327
--builtin_ntp_servers=127.20.113.20:35497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260806 09:29:54.367308 2218 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:29:54.367589 2218 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:29:54.367679 2218 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:29:54.374809 2218 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:29:54.374929 2218 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:29:54.375036 2218 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.113.1
I20260806 09:29:54.384033 2218 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:35497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786008495763054-20932-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786008495763054-20932-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.113.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786008495763054-20932-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.113.1
--webserver_port=0
--tserver_master_addrs=127.20.113.62:45327
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.113.1
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786008495763054-20932-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:29:54.385315 2218 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:29:54.387037 2218 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:29:54.400480 2228 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:29:54.400810 2224 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:29:54.401120 2225 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:29:54.401343 2218 server_base.cc:1034] running on GCE node
I20260806 09:29:54.402021 2218 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:29:54.403010 2218 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:29:54.404260 2218 hybrid_clock.cc:648] HybridClock initialized: now 1786008594404203 us; error 50 us; skew 500 ppm
I20260806 09:29:54.404619 2218 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 09:29:54.406981 2218 webserver.cc:460] Webserver started at http://127.20.113.1:42429/ using document root <none> and password file <none>
I20260806 09:29:54.407694 2218 fs_manager.cc:359] Metadata directory not provided
I20260806 09:29:54.407840 2218 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:29:54.408183 2218 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:29:54.411041 2218 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786008495763054-20932-0/minicluster-data/ts-0/data/instance:
uuid: "09b8506cbf2c4376b5f2862c1ee92401"
format_stamp: "Formatted at 2026-08-06 09:29:54 on dist-test-slave-45dx"
server_key: "314942bbd2203ca9d612fa6e3e2fd925"
server_key_iv: "43ff7ed5cdabec8f4ad916a2c3271cbf"
server_key_version: "encryptionkey@0"
I20260806 09:29:54.411814 2218 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786008495763054-20932-0/minicluster-data/ts-0/wal/instance:
uuid: "09b8506cbf2c4376b5f2862c1ee92401"
format_stamp: "Formatted at 2026-08-06 09:29:54 on dist-test-slave-45dx"
server_key: "314942bbd2203ca9d612fa6e3e2fd925"
server_key_iv: "43ff7ed5cdabec8f4ad916a2c3271cbf"
server_key_version: "encryptionkey@0"
I20260806 09:29:54.416853 2218 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260806 09:29:54.420632 2237 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:29:54.421974 2218 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 09:29:54.422165 2218 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786008495763054-20932-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786008495763054-20932-0/minicluster-data/ts-0/wal
uuid: "09b8506cbf2c4376b5f2862c1ee92401"
format_stamp: "Formatted at 2026-08-06 09:29:54 on dist-test-slave-45dx"
server_key: "314942bbd2203ca9d612fa6e3e2fd925"
server_key_iv: "43ff7ed5cdabec8f4ad916a2c3271cbf"
server_key_version: "encryptionkey@0"
I20260806 09:29:54.422390 2218 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786008495763054-20932-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786008495763054-20932-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786008495763054-20932-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:29:54.436661 2218 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:29:54.437491 2218 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:29:54.437853 2218 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:29:54.438897 2218 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:29:54.440405 2218 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:29:54.440516 2218 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:29:54.440601 2218 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:29:54.440657 2218 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:29:54.474650 2218 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.1:34709
I20260806 09:29:54.474684 2401 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.1:34709 every 8 connection(s)
I20260806 09:29:54.476004 2218 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786008495763054-20932-0/minicluster-data/ts-0/data/info.pb
I20260806 09:29:54.484717 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 2218
I20260806 09:29:54.484920 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786008495763054-20932-0/minicluster-data/ts-0/wal/instance
I20260806 09:29:54.485594 20932 external_mini_cluster.cc:1428] Setting key 1b636891f80a1683fc38d0441405f30f
I20260806 09:29:54.488404 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786008495763054-20932-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786008495763054-20932-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786008495763054-20932-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786008495763054-20932-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.113.2:0
--local_ip_for_outbound_sockets=127.20.113.2
--webserver_interface=127.20.113.2
--webserver_port=0
--tserver_master_addrs=127.20.113.62:45327
--builtin_ntp_servers=127.20.113.20:35497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260806 09:29:54.488925 2405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:54.478591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.1:39675 (local address 127.20.113.62:45327)
0806 09:29:54.479041 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:54.479045 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:54.480244 (+ 1199us) server_negotiation.cc:407] Connection header received
0806 09:29:54.482125 (+ 1881us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:54.482131 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:54.482219 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:54.482349 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:54.483427 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:54.484149 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:54.485935 (+ 1786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:54.486477 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:54.486620 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:54.487999 (+ 1379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:29:54.488005 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:29:54.488010 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:29:54.488033 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:29:54.488073 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:29:54.488085 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:29:54.488184 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:54.488476 (+ 292us) server_negotiation.cc:299] Negotiation successful
0806 09:29:54.488570 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":338,"thread_start_us":221,"threads_started":1}
I20260806 09:29:54.489840 2406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:54.479189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:45327 (local address 127.20.113.1:39675)
0806 09:29:54.480077 (+ 888us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:54.480113 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:54.481854 (+ 1741us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:54.482602 (+ 748us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:54.482610 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:54.482692 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:29:54.483250 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:54.483269 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:54.484345 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:54.484350 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:54.485777 (+ 1427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:54.485790 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:54.486709 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:54.486717 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:54.486952 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:54.487684 (+ 732us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 09:29:54.487686 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:29:54.487725 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 09:29:54.487797 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:29:54.488186 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:29:54.488200 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:29:54.488209 (+ 9us) client_negotiation.cc:769] Sending connection context
0806 09:29:54.488466 (+ 257us) client_negotiation.cc:240] Negotiation successful
0806 09:29:54.488835 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":291,"threads_started":1}
I20260806 09:29:54.491236 2403 heartbeater.cc:347] Connected to a master server at 127.20.113.62:45327
I20260806 09:29:54.491533 2403 heartbeater.cc:464] Registering TS with master...
I20260806 09:29:54.492265 2403 heartbeater.cc:511] Master 127.20.113.62:45327 requested a full tablet report, sending...
I20260806 09:29:54.494303 2128 ts_manager.cc:194] Registered new tserver with Master: 09b8506cbf2c4376b5f2862c1ee92401 (127.20.113.1:34709)
I20260806 09:29:54.495764 2128 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.113.1:39675
W20260806 09:29:54.834069 2408 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:29:54.834542 2408 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:29:54.834681 2408 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:29:54.846204 2408 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:29:54.846309 2408 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:29:54.846499 2408 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.113.2
I20260806 09:29:54.855966 2408 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:35497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786008495763054-20932-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786008495763054-20932-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.113.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786008495763054-20932-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.113.2
--webserver_port=0
--tserver_master_addrs=127.20.113.62:45327
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.113.2
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786008495763054-20932-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:29:54.857350 2408 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:29:54.859200 2408 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:29:54.872676 2415 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:29:54.872815 2413 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:29:54.872778 2419 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:29:54.873731 2408 server_base.cc:1034] running on GCE node
I20260806 09:29:54.874357 2408 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:29:54.875295 2408 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:29:54.876533 2408 hybrid_clock.cc:648] HybridClock initialized: now 1786008594876503 us; error 68 us; skew 500 ppm
I20260806 09:29:54.876860 2408 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 09:29:54.879179 2408 webserver.cc:460] Webserver started at http://127.20.113.2:43013/ using document root <none> and password file <none>
I20260806 09:29:54.879878 2408 fs_manager.cc:359] Metadata directory not provided
I20260806 09:29:54.880028 2408 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:29:54.880470 2408 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:29:54.883224 2408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786008495763054-20932-0/minicluster-data/ts-1/data/instance:
uuid: "78ca38a733ed4514b5bbd58022c25fd7"
format_stamp: "Formatted at 2026-08-06 09:29:54 on dist-test-slave-45dx"
server_key: "37946bd9ccdc3dc077fb2d941b049883"
server_key_iv: "82f8531b24781c6441c0ce32805d76ab"
server_key_version: "encryptionkey@0"
I20260806 09:29:54.884034 2408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786008495763054-20932-0/minicluster-data/ts-1/wal/instance:
uuid: "78ca38a733ed4514b5bbd58022c25fd7"
format_stamp: "Formatted at 2026-08-06 09:29:54 on dist-test-slave-45dx"
server_key: "37946bd9ccdc3dc077fb2d941b049883"
server_key_iv: "82f8531b24781c6441c0ce32805d76ab"
server_key_version: "encryptionkey@0"
I20260806 09:29:54.889147 2408 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260806 09:29:54.892690 2425 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:29:54.894024 2408 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260806 09:29:54.894209 2408 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786008495763054-20932-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786008495763054-20932-0/minicluster-data/ts-1/wal
uuid: "78ca38a733ed4514b5bbd58022c25fd7"
format_stamp: "Formatted at 2026-08-06 09:29:54 on dist-test-slave-45dx"
server_key: "37946bd9ccdc3dc077fb2d941b049883"
server_key_iv: "82f8531b24781c6441c0ce32805d76ab"
server_key_version: "encryptionkey@0"
I20260806 09:29:54.894434 2408 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786008495763054-20932-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786008495763054-20932-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786008495763054-20932-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:29:54.917187 2408 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:29:54.918141 2408 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:29:54.918493 2408 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:29:54.919512 2408 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:29:54.921173 2408 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:29:54.921300 2408 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:29:54.921437 2408 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:29:54.921530 2408 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:29:54.956485 2408 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.2:43537
I20260806 09:29:54.956545 2590 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.2:43537 every 8 connection(s)
I20260806 09:29:54.957823 2408 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786008495763054-20932-0/minicluster-data/ts-1/data/info.pb
I20260806 09:29:54.959525 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 2408
I20260806 09:29:54.959702 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786008495763054-20932-0/minicluster-data/ts-1/wal/instance
I20260806 09:29:54.960340 20932 external_mini_cluster.cc:1428] Setting key 1dbe41f3e6f617ea5dd107be312eb2a9
I20260806 09:29:54.963559 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786008495763054-20932-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786008495763054-20932-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786008495763054-20932-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786008495763054-20932-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.113.3:0
--local_ip_for_outbound_sockets=127.20.113.3
--webserver_interface=127.20.113.3
--webserver_port=0
--tserver_master_addrs=127.20.113.62:45327
--builtin_ntp_servers=127.20.113.20:35497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260806 09:29:54.971695 2405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:54.960682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.2:41191 (local address 127.20.113.62:45327)
0806 09:29:54.960879 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:54.960884 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:54.962445 (+ 1561us) server_negotiation.cc:407] Connection header received
0806 09:29:54.964543 (+ 2098us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:54.964551 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:54.964646 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:54.964764 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:54.965813 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:54.966958 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:54.968569 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:54.969273 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:54.969431 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:54.970772 (+ 1341us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:29:54.970781 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:29:54.970789 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:29:54.970818 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:29:54.970873 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:29:54.970891 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:29:54.970985 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:54.971189 (+ 204us) server_negotiation.cc:299] Negotiation successful
0806 09:29:54.971298 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 09:29:54.972635 2594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:54.961265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:45327 (local address 127.20.113.2:41191)
0806 09:29:54.962252 (+ 987us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:54.962297 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:54.964186 (+ 1889us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:54.964962 (+ 776us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:54.964970 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:54.965053 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:29:54.965631 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:54.965648 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:54.967111 (+ 1463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:54.967115 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:54.968413 (+ 1298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:54.968426 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:54.969491 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:54.969496 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:54.969693 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:54.970445 (+ 752us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 09:29:54.970447 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:29:54.970486 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 09:29:54.970568 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:29:54.970991 (+ 423us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:29:54.971000 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:29:54.971005 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 09:29:54.971145 (+ 140us) client_negotiation.cc:240] Negotiation successful
0806 09:29:54.971430 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":547,"thread_start_us":347,"threads_started":1}
I20260806 09:29:54.974160 2592 heartbeater.cc:347] Connected to a master server at 127.20.113.62:45327
I20260806 09:29:54.974488 2592 heartbeater.cc:464] Registering TS with master...
I20260806 09:29:54.975224 2592 heartbeater.cc:511] Master 127.20.113.62:45327 requested a full tablet report, sending...
I20260806 09:29:54.976739 2128 ts_manager.cc:194] Registered new tserver with Master: 78ca38a733ed4514b5bbd58022c25fd7 (127.20.113.2:43537)
I20260806 09:29:54.977658 2128 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.113.2:41191
W20260806 09:29:55.325071 2595 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:29:55.325413 2595 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:29:55.325577 2595 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:29:55.333091 2595 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:29:55.333246 2595 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:29:55.333427 2595 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.113.3
I20260806 09:29:55.342603 2595 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:35497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786008495763054-20932-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786008495763054-20932-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.113.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786008495763054-20932-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.113.3
--webserver_port=0
--tserver_master_addrs=127.20.113.62:45327
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.113.3
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786008495763054-20932-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:29:55.343919 2595 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:29:55.345749 2595 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:29:55.359513 2603 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:29:55.359550 2605 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:29:55.359513 2602 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:29:55.360658 2595 server_base.cc:1034] running on GCE node
I20260806 09:29:55.361353 2595 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:29:55.362383 2595 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:29:55.363623 2595 hybrid_clock.cc:648] HybridClock initialized: now 1786008595363566 us; error 67 us; skew 500 ppm
I20260806 09:29:55.363963 2595 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 09:29:55.366321 2595 webserver.cc:460] Webserver started at http://127.20.113.3:44049/ using document root <none> and password file <none>
I20260806 09:29:55.367089 2595 fs_manager.cc:359] Metadata directory not provided
I20260806 09:29:55.367235 2595 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:29:55.367604 2595 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:29:55.370565 2595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786008495763054-20932-0/minicluster-data/ts-2/data/instance:
uuid: "5b8dd0ac2a9e4e90a58970022b1fa2e5"
format_stamp: "Formatted at 2026-08-06 09:29:55 on dist-test-slave-45dx"
server_key: "621093fe908b7d985ad4caa0ca38c377"
server_key_iv: "8264dad518df2513cc91d7c91ae524ac"
server_key_version: "encryptionkey@0"
I20260806 09:29:55.371387 2595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786008495763054-20932-0/minicluster-data/ts-2/wal/instance:
uuid: "5b8dd0ac2a9e4e90a58970022b1fa2e5"
format_stamp: "Formatted at 2026-08-06 09:29:55 on dist-test-slave-45dx"
server_key: "621093fe908b7d985ad4caa0ca38c377"
server_key_iv: "8264dad518df2513cc91d7c91ae524ac"
server_key_version: "encryptionkey@0"
I20260806 09:29:55.376547 2595 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 09:29:55.379848 2615 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:29:55.381096 2595 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 09:29:55.381278 2595 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786008495763054-20932-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786008495763054-20932-0/minicluster-data/ts-2/wal
uuid: "5b8dd0ac2a9e4e90a58970022b1fa2e5"
format_stamp: "Formatted at 2026-08-06 09:29:55 on dist-test-slave-45dx"
server_key: "621093fe908b7d985ad4caa0ca38c377"
server_key_iv: "8264dad518df2513cc91d7c91ae524ac"
server_key_version: "encryptionkey@0"
I20260806 09:29:55.381508 2595 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786008495763054-20932-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786008495763054-20932-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786008495763054-20932-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:29:55.408041 2595 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:29:55.408963 2595 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:29:55.409276 2595 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:29:55.410231 2595 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:29:55.411638 2595 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:29:55.411767 2595 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:29:55.411896 2595 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:29:55.411994 2595 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:29:55.446985 2595 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.3:40429
I20260806 09:29:55.447054 2785 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.3:40429 every 8 connection(s)
I20260806 09:29:55.448529 2595 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786008495763054-20932-0/minicluster-data/ts-2/data/info.pb
I20260806 09:29:55.449030 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 2595
I20260806 09:29:55.449241 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786008495763054-20932-0/minicluster-data/ts-2/wal/instance
I20260806 09:29:55.449963 20932 external_mini_cluster.cc:1428] Setting key 483ab9d4baa157b270fee08ae012e95d
I20260806 09:29:55.462071 2405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:55.451685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.3:51531 (local address 127.20.113.62:45327)
0806 09:29:55.451896 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:55.451906 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:55.453621 (+ 1715us) server_negotiation.cc:407] Connection header received
0806 09:29:55.455671 (+ 2050us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:55.455675 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:55.455743 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:55.455826 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:55.456940 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:55.457685 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:55.459183 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:55.459578 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:55.459681 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:55.461257 (+ 1576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:29:55.461262 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:29:55.461265 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:29:55.461285 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:29:55.461321 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:29:55.461334 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:29:55.461413 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:55.461693 (+ 280us) server_negotiation.cc:299] Negotiation successful
0806 09:29:55.461800 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 09:29:55.463083 2790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:55.452262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:45327 (local address 127.20.113.3:51531)
0806 09:29:55.453417 (+ 1155us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:55.453488 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:55.455420 (+ 1932us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:55.455993 (+ 573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:55.456001 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:55.456096 (+ 95us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:29:55.456784 (+ 688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:55.456801 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:55.457817 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:55.457821 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:55.459047 (+ 1226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:29:55.459061 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:29:55.459789 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:29:55.459794 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:29:55.459983 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:55.460712 (+ 729us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 09:29:55.460715 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:29:55.460754 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 09:29:55.461020 (+ 266us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:29:55.461439 (+ 419us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:29:55.461449 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:29:55.461455 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 09:29:55.461609 (+ 154us) client_negotiation.cc:240] Negotiation successful
0806 09:29:55.461953 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":626,"thread_start_us":333,"threads_started":1}
I20260806 09:29:55.464699 2786 heartbeater.cc:347] Connected to a master server at 127.20.113.62:45327
I20260806 09:29:55.465085 2786 heartbeater.cc:464] Registering TS with master...
I20260806 09:29:55.466029 2786 heartbeater.cc:511] Master 127.20.113.62:45327 requested a full tablet report, sending...
I20260806 09:29:55.467445 2128 ts_manager.cc:194] Registered new tserver with Master: 5b8dd0ac2a9e4e90a58970022b1fa2e5 (127.20.113.3:40429)
I20260806 09:29:55.468525 2128 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.113.3:51531
I20260806 09:29:55.478590 20932 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 09:29:55.484747 2405 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 09:29:55.483197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55288 (local address 127.20.113.62:45327)
0806 09:29:55.483369 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:55.483374 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:55.483871 (+ 497us) server_negotiation.cc:407] Connection header received
0806 09:29:55.483985 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:55.483989 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:55.484053 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:55.484127 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:55.484579 (+ 452us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55288: BlockingRecv error: recv got EOF from 127.0.0.1:55288 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 09:29:55.485317 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 2218
I20260806 09:29:55.493371 2798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:55.486788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50404 (local address 127.20.113.1:34709)
0806 09:29:55.487360 (+ 572us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:55.487367 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:55.487390 (+ 23us) server_negotiation.cc:407] Connection header received
0806 09:29:55.487450 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:55.487455 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:55.487630 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:55.487749 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:55.488089 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:55.489150 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:55.490383 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:55.490934 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:55.491032 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:55.492020 (+ 988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:29:55.492027 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:29:55.492038 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:29:55.492669 (+ 631us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:29:55.492715 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:29:55.492735 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:29:55.492835 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:55.492948 (+ 113us) server_negotiation.cc:299] Negotiation successful
0806 09:29:55.493056 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":402,"thread_start_us":294,"threads_started":1}
I20260806 09:29:55.494622 2396 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:29:55.498521 2403 heartbeater.cc:503] Master 127.20.113.62:45327 was elected leader, sending a full tablet report...
I20260806 09:29:55.576980 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 2218
W20260806 09:29:55.602301 2109 connection.cc:535] server connection from 127.20.113.1:39675 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 09:29:55.602738 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 2408
I20260806 09:29:55.610526 2802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:55.603722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55290 (local address 127.20.113.2:43537)
0806 09:29:55.604340 (+ 618us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:55.604351 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:55.604384 (+ 33us) server_negotiation.cc:407] Connection header received
0806 09:29:55.604446 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:55.604453 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:55.604671 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:55.604802 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:55.605187 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:55.606195 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:55.607379 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:55.607969 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:55.608063 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:55.609179 (+ 1116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:29:55.609187 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:29:55.609199 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:29:55.609784 (+ 585us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:29:55.609831 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:29:55.609850 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:29:55.609954 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:55.610102 (+ 148us) server_negotiation.cc:299] Negotiation successful
0806 09:29:55.610211 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":426,"thread_start_us":317,"threads_started":1}
I20260806 09:29:55.611737 2583 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:29:55.694149 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 2408
I20260806 09:29:55.721009 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 2595
I20260806 09:29:55.729771 2804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:55.722090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50252 (local address 127.20.113.3:40429)
0806 09:29:55.722781 (+ 691us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:55.722787 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:55.722819 (+ 32us) server_negotiation.cc:407] Connection header received
0806 09:29:55.722882 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:55.722889 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:55.723172 (+ 283us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:55.723354 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:55.723881 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:55.725074 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:55.726349 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:29:55.726924 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:29:55.727083 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:29:55.728197 (+ 1114us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:29:55.728206 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:29:55.728219 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:29:55.728871 (+ 652us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:29:55.728919 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:29:55.728949 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:29:55.729105 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:55.729251 (+ 146us) server_negotiation.cc:299] Negotiation successful
0806 09:29:55.729383 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":541,"thread_start_us":367,"threads_started":1}
I20260806 09:29:55.731081 2780 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:29:55.815358 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 2595
I20260806 09:29:55.837781 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 2083
I20260806 09:29:55.839124 2169 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:29:55.943392 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 2083
2026-08-06T09:29:55Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2616 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-06T09:29:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T09:29:56Z Disabled control of system clock
I20260806 09:29:56.043182 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786008495763054-20932-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786008495763054-20932-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786008495763054-20932-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.113.62:42643
--webserver_interface=127.20.113.62
--webserver_port=0
--builtin_ntp_servers=127.20.113.20:46749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.113.62:42643
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 09:29:56.386613 2814 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:29:56.386955 2814 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:29:56.387125 2814 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:29:56.394121 2814 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 09:29:56.394248 2814 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:29:56.394335 2814 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:29:56.394433 2814 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 09:29:56.394510 2814 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 09:29:56.403367 2814 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:46749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786008495763054-20932-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786008495763054-20932-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.113.62:42643
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.113.62:42643
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.113.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786008495763054-20932-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:29:56.404626 2814 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:29:56.406445 2814 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:29:56.418336 2824 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:29:56.418403 2827 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:29:56.418444 2821 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:29:56.419242 2814 server_base.cc:1034] running on GCE node
I20260806 09:29:56.420158 2814 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:29:56.421522 2814 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:29:56.422824 2814 hybrid_clock.cc:648] HybridClock initialized: now 1786008596422782 us; error 43 us; skew 500 ppm
I20260806 09:29:56.423313 2814 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 09:29:56.425662 2814 webserver.cc:460] Webserver started at http://127.20.113.62:42889/ using document root <none> and password file <none>
I20260806 09:29:56.426378 2814 fs_manager.cc:359] Metadata directory not provided
I20260806 09:29:56.426515 2814 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:29:56.426868 2814 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:29:56.432088 2814 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786008495763054-20932-0/minicluster-data/master-0/data/instance:
uuid: "6baad320e1944fb7bd2776628ecd685f"
format_stamp: "Formatted at 2026-08-06 09:29:56 on dist-test-slave-45dx"
server_key: "35252fcb2192632234e7de102e027fd6"
server_key_iv: "2540beea5667573b9662ed902f1af36b"
server_key_version: "encryptionkey@0"
I20260806 09:29:56.433027 2814 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786008495763054-20932-0/minicluster-data/master-0/wal/instance:
uuid: "6baad320e1944fb7bd2776628ecd685f"
format_stamp: "Formatted at 2026-08-06 09:29:56 on dist-test-slave-45dx"
server_key: "35252fcb2192632234e7de102e027fd6"
server_key_iv: "2540beea5667573b9662ed902f1af36b"
server_key_version: "encryptionkey@0"
I20260806 09:29:56.438175 2814 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 09:29:56.441663 2838 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:29:56.442948 2814 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 09:29:56.443140 2814 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786008495763054-20932-0/minicluster-data/master-0/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786008495763054-20932-0/minicluster-data/master-0/wal
uuid: "6baad320e1944fb7bd2776628ecd685f"
format_stamp: "Formatted at 2026-08-06 09:29:56 on dist-test-slave-45dx"
server_key: "35252fcb2192632234e7de102e027fd6"
server_key_iv: "2540beea5667573b9662ed902f1af36b"
server_key_version: "encryptionkey@0"
I20260806 09:29:56.443373 2814 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786008495763054-20932-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786008495763054-20932-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786008495763054-20932-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:29:56.449669 2814 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:29:56.450563 2814 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:29:56.450878 2814 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:29:56.470091 2814 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.62:42643
I20260806 09:29:56.470125 2910 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.62:42643 every 8 connection(s)
I20260806 09:29:56.471552 2814 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
I20260806 09:29:56.475334 2912 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:29:56.481529 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 2814
I20260806 09:29:56.481772 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786008495763054-20932-0/minicluster-data/master-0/wal/instance
I20260806 09:29:56.482465 20932 external_mini_cluster.cc:1428] Setting key 1f0f05e10bb849081ecdf43a042855fc
I20260806 09:29:56.487408 2912 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6baad320e1944fb7bd2776628ecd685f: Bootstrap starting.
I20260806 09:29:56.491995 2912 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6baad320e1944fb7bd2776628ecd685f: Neither blocks nor log segments found. Creating new log.
I20260806 09:29:56.493165 2916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:56.484743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55444 (local address 127.20.113.62:42643)
0806 09:29:56.485924 (+ 1181us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:56.485940 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:56.485997 (+ 57us) server_negotiation.cc:407] Connection header received
0806 09:29:56.488647 (+ 2650us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:56.488677 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:56.489234 (+ 557us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:56.489768 (+ 534us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:56.490883 (+ 1115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:29:56.490892 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:29:56.490909 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:29:56.490951 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:29:56.491035 (+ 84us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:29:56.491060 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:29:56.491255 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:56.491404 (+ 149us) server_negotiation.cc:299] Negotiation successful
0806 09:29:56.491686 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":619,"thread_start_us":331,"threads_started":1}
I20260806 09:29:56.493571 2912 log.cc:824] T 00000000000000000000000000000000 P 6baad320e1944fb7bd2776628ecd685f: Log is configured to *not* fsync() on all Append() calls
I20260806 09:29:56.497216 2912 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6baad320e1944fb7bd2776628ecd685f: No bootstrap required, opened a new log
I20260806 09:29:56.505713 2912 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6baad320e1944fb7bd2776628ecd685f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6baad320e1944fb7bd2776628ecd685f" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 42643 } }
I20260806 09:29:56.506145 2912 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6baad320e1944fb7bd2776628ecd685f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:29:56.506271 2912 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6baad320e1944fb7bd2776628ecd685f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6baad320e1944fb7bd2776628ecd685f, State: Initialized, Role: FOLLOWER
I20260806 09:29:56.506877 2912 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6baad320e1944fb7bd2776628ecd685f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6baad320e1944fb7bd2776628ecd685f" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 42643 } }
I20260806 09:29:56.507158 2912 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6baad320e1944fb7bd2776628ecd685f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 09:29:56.507324 2912 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6baad320e1944fb7bd2776628ecd685f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 09:29:56.507526 2912 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6baad320e1944fb7bd2776628ecd685f [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:29:56.510296 2912 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6baad320e1944fb7bd2776628ecd685f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6baad320e1944fb7bd2776628ecd685f" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 42643 } }
I20260806 09:29:56.510812 2912 leader_election.cc:302] T 00000000000000000000000000000000 P 6baad320e1944fb7bd2776628ecd685f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6baad320e1944fb7bd2776628ecd685f; no voters:
I20260806 09:29:56.511453 2912 leader_election.cc:288] T 00000000000000000000000000000000 P 6baad320e1944fb7bd2776628ecd685f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 09:29:56.511602 2918 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6baad320e1944fb7bd2776628ecd685f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:29:56.512403 2918 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6baad320e1944fb7bd2776628ecd685f [term 1 LEADER]: Becoming Leader. State: Replica: 6baad320e1944fb7bd2776628ecd685f, State: Running, Role: LEADER
I20260806 09:29:56.512989 2918 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6baad320e1944fb7bd2776628ecd685f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6baad320e1944fb7bd2776628ecd685f" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 42643 } }
I20260806 09:29:56.513787 2912 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6baad320e1944fb7bd2776628ecd685f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 09:29:56.515949 2921 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6baad320e1944fb7bd2776628ecd685f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6baad320e1944fb7bd2776628ecd685f. Latest consensus state: current_term: 1 leader_uuid: "6baad320e1944fb7bd2776628ecd685f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6baad320e1944fb7bd2776628ecd685f" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 42643 } } }
I20260806 09:29:56.516537 2921 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6baad320e1944fb7bd2776628ecd685f [sys.catalog]: This master's current role is: LEADER
I20260806 09:29:56.516324 2920 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6baad320e1944fb7bd2776628ecd685f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6baad320e1944fb7bd2776628ecd685f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6baad320e1944fb7bd2776628ecd685f" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 42643 } } }
I20260806 09:29:56.516683 2920 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6baad320e1944fb7bd2776628ecd685f [sys.catalog]: This master's current role is: LEADER
I20260806 09:29:56.518405 2928 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 09:29:56.523706 2928 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 09:29:56.532701 2928 catalog_manager.cc:1347] Generated new cluster ID: 1c1a8c34bc454781bcd3bfd266a2365a
I20260806 09:29:56.532809 2928 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 09:29:56.548166 2928 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 09:29:56.548664 2928 catalog_manager.cc:1504] Loading token signing keys...
I20260806 09:29:56.565302 2928 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6baad320e1944fb7bd2776628ecd685f: Generated new TSK 0
I20260806 09:29:56.566103 2928 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 09:29:56.580860 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786008495763054-20932-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786008495763054-20932-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786008495763054-20932-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786008495763054-20932-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.113.1:0
--local_ip_for_outbound_sockets=127.20.113.1
--webserver_interface=127.20.113.1
--webserver_port=0
--tserver_master_addrs=127.20.113.62:42643
--builtin_ntp_servers=127.20.113.20:46749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 09:29:56.953904 2945 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:29:56.954218 2945 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:29:56.954391 2945 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:29:56.961681 2945 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:29:56.961853 2945 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:29:56.962057 2945 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.113.1
I20260806 09:29:56.971251 2945 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:46749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786008495763054-20932-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786008495763054-20932-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.113.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786008495763054-20932-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.113.1
--webserver_port=0
--tserver_master_addrs=127.20.113.62:42643
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.113.1
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786008495763054-20932-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:29:56.972631 2945 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:29:56.974524 2945 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:29:56.988382 2950 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:29:56.988374 2951 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:29:56.988690 2956 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:29:56.993328 2945 server_base.cc:1034] running on GCE node
I20260806 09:29:56.993953 2945 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:29:56.995023 2945 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:29:56.996287 2945 hybrid_clock.cc:648] HybridClock initialized: now 1786008596996233 us; error 44 us; skew 500 ppm
I20260806 09:29:56.996662 2945 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 09:29:56.999161 2945 webserver.cc:460] Webserver started at http://127.20.113.1:33703/ using document root <none> and password file <none>
I20260806 09:29:56.999884 2945 fs_manager.cc:359] Metadata directory not provided
I20260806 09:29:57.000032 2945 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:29:57.000468 2945 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:29:57.003396 2945 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786008495763054-20932-0/minicluster-data/ts-0/data/instance:
uuid: "7c4b3000a2fb42af8949c0ceec0db9a9"
format_stamp: "Formatted at 2026-08-06 09:29:56 on dist-test-slave-45dx"
server_key: "2541af3a3ef335708074cb36e02e82a3"
server_key_iv: "f03eae6ec4251a1f3afc00a27a389c6f"
server_key_version: "encryptionkey@0"
I20260806 09:29:57.004185 2945 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786008495763054-20932-0/minicluster-data/ts-0/wal/instance:
uuid: "7c4b3000a2fb42af8949c0ceec0db9a9"
format_stamp: "Formatted at 2026-08-06 09:29:56 on dist-test-slave-45dx"
server_key: "2541af3a3ef335708074cb36e02e82a3"
server_key_iv: "f03eae6ec4251a1f3afc00a27a389c6f"
server_key_version: "encryptionkey@0"
I20260806 09:29:57.009372 2945 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260806 09:29:57.012907 2964 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:29:57.014215 2945 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 09:29:57.014391 2945 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786008495763054-20932-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786008495763054-20932-0/minicluster-data/ts-0/wal
uuid: "7c4b3000a2fb42af8949c0ceec0db9a9"
format_stamp: "Formatted at 2026-08-06 09:29:56 on dist-test-slave-45dx"
server_key: "2541af3a3ef335708074cb36e02e82a3"
server_key_iv: "f03eae6ec4251a1f3afc00a27a389c6f"
server_key_version: "encryptionkey@0"
I20260806 09:29:57.014617 2945 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786008495763054-20932-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786008495763054-20932-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786008495763054-20932-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:29:57.020275 2945 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:29:57.021108 2945 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:29:57.021386 2945 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:29:57.022310 2945 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:29:57.023679 2945 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:29:57.023806 2945 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:29:57.023940 2945 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:29:57.024032 2945 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:29:57.058182 2945 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.1:35727
I20260806 09:29:57.058255 3126 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.1:35727 every 8 connection(s)
I20260806 09:29:57.059561 2945 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786008495763054-20932-0/minicluster-data/ts-0/data/info.pb
I20260806 09:29:57.060024 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 2945
I20260806 09:29:57.060240 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786008495763054-20932-0/minicluster-data/ts-0/wal/instance
I20260806 09:29:57.061179 20932 external_mini_cluster.cc:1428] Setting key 0f6b851014d91f5aaa5ee11cca04a889
I20260806 09:29:57.064424 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786008495763054-20932-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786008495763054-20932-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786008495763054-20932-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786008495763054-20932-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.113.2:0
--local_ip_for_outbound_sockets=127.20.113.2
--webserver_interface=127.20.113.2
--webserver_port=0
--tserver_master_addrs=127.20.113.62:42643
--builtin_ntp_servers=127.20.113.20:46749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260806 09:29:57.069690 3129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:57.062421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.1:46147 (local address 127.20.113.62:42643)
0806 09:29:57.062925 (+ 504us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:57.062932 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:57.064389 (+ 1457us) server_negotiation.cc:407] Connection header received
0806 09:29:57.067201 (+ 2812us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:57.067205 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:57.067293 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:57.067459 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:57.069007 (+ 1548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:29:57.069011 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:29:57.069014 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:29:57.069029 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:29:57.069072 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:29:57.069083 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:29:57.069177 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:57.069415 (+ 238us) server_negotiation.cc:299] Negotiation successful
0806 09:29:57.069464 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":252,"threads_started":1}
I20260806 09:29:57.070875 3130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:57.063100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:42643 (local address 127.20.113.1:46147)
0806 09:29:57.064209 (+ 1109us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:57.064258 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:57.066884 (+ 2626us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:57.067679 (+ 795us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:57.067693 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:57.067809 (+ 116us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:29:57.068625 (+ 816us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 09:29:57.068631 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:29:57.068684 (+ 53us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 09:29:57.068792 (+ 108us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:29:57.069181 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:29:57.069194 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:29:57.069203 (+ 9us) client_negotiation.cc:769] Sending connection context
0806 09:29:57.069491 (+ 288us) client_negotiation.cc:240] Negotiation successful
0806 09:29:57.069741 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":525,"thread_start_us":292,"threads_started":1}
I20260806 09:29:57.072361 3127 heartbeater.cc:347] Connected to a master server at 127.20.113.62:42643
I20260806 09:29:57.072664 3127 heartbeater.cc:464] Registering TS with master...
I20260806 09:29:57.073226 3127 heartbeater.cc:511] Master 127.20.113.62:42643 requested a full tablet report, sending...
I20260806 09:29:57.075358 2861 ts_manager.cc:194] Registered new tserver with Master: 7c4b3000a2fb42af8949c0ceec0db9a9 (127.20.113.1:35727)
W20260806 09:29:57.404138 3131 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:29:57.404429 3131 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:29:57.404625 3131 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:29:57.411569 3131 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:29:57.411672 3131 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:29:57.411844 3131 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.113.2
I20260806 09:29:57.421026 3131 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:46749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786008495763054-20932-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786008495763054-20932-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.113.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786008495763054-20932-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.113.2
--webserver_port=0
--tserver_master_addrs=127.20.113.62:42643
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.113.2
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786008495763054-20932-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:29:57.422325 3131 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:29:57.424098 3131 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:29:57.437186 3137 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:29:57.437300 3136 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:29:57.437331 3141 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:29:57.438230 3131 server_base.cc:1034] running on GCE node
I20260806 09:29:57.438774 3131 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:29:57.439642 3131 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:29:57.440867 3131 hybrid_clock.cc:648] HybridClock initialized: now 1786008597440831 us; error 31 us; skew 500 ppm
I20260806 09:29:57.441139 3131 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 09:29:57.443409 3131 webserver.cc:460] Webserver started at http://127.20.113.2:33495/ using document root <none> and password file <none>
I20260806 09:29:57.444080 3131 fs_manager.cc:359] Metadata directory not provided
I20260806 09:29:57.444168 3131 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:29:57.444641 3131 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:29:57.447652 3131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786008495763054-20932-0/minicluster-data/ts-1/data/instance:
uuid: "640bf4f13e174bcfb2a3d239ab4adc8a"
format_stamp: "Formatted at 2026-08-06 09:29:57 on dist-test-slave-45dx"
server_key: "67f213562c54ee57db907ea58c30af60"
server_key_iv: "4ac44beb835a25322dfbeb2915c01cc7"
server_key_version: "encryptionkey@0"
I20260806 09:29:57.448565 3131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786008495763054-20932-0/minicluster-data/ts-1/wal/instance:
uuid: "640bf4f13e174bcfb2a3d239ab4adc8a"
format_stamp: "Formatted at 2026-08-06 09:29:57 on dist-test-slave-45dx"
server_key: "67f213562c54ee57db907ea58c30af60"
server_key_iv: "4ac44beb835a25322dfbeb2915c01cc7"
server_key_version: "encryptionkey@0"
I20260806 09:29:57.454125 3131 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 09:29:57.457563 3150 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:29:57.458959 3131 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 09:29:57.459131 3131 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786008495763054-20932-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786008495763054-20932-0/minicluster-data/ts-1/wal
uuid: "640bf4f13e174bcfb2a3d239ab4adc8a"
format_stamp: "Formatted at 2026-08-06 09:29:57 on dist-test-slave-45dx"
server_key: "67f213562c54ee57db907ea58c30af60"
server_key_iv: "4ac44beb835a25322dfbeb2915c01cc7"
server_key_version: "encryptionkey@0"
I20260806 09:29:57.459383 3131 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786008495763054-20932-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786008495763054-20932-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786008495763054-20932-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:29:57.465085 3131 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:29:57.465811 3131 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:29:57.466038 3131 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:29:57.466948 3131 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:29:57.468277 3131 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:29:57.468437 3131 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:29:57.468525 3131 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:29:57.468582 3131 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:29:57.503962 3131 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.2:45449
I20260806 09:29:57.504019 3322 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.2:45449 every 8 connection(s)
I20260806 09:29:57.505448 3131 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786008495763054-20932-0/minicluster-data/ts-1/data/info.pb
I20260806 09:29:57.507615 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 3131
I20260806 09:29:57.507826 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786008495763054-20932-0/minicluster-data/ts-1/wal/instance
I20260806 09:29:57.508424 20932 external_mini_cluster.cc:1428] Setting key 4dd8397c067ec47df1ba548fa61a854a
I20260806 09:29:57.511523 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786008495763054-20932-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786008495763054-20932-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786008495763054-20932-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786008495763054-20932-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.113.3:0
--local_ip_for_outbound_sockets=127.20.113.3
--webserver_interface=127.20.113.3
--webserver_port=0
--tserver_master_addrs=127.20.113.62:42643
--builtin_ntp_servers=127.20.113.20:46749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260806 09:29:57.514540 3129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:57.508285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.2:41581 (local address 127.20.113.62:42643)
0806 09:29:57.508516 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:57.508523 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:57.510081 (+ 1558us) server_negotiation.cc:407] Connection header received
0806 09:29:57.512091 (+ 2010us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:57.512099 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:57.512178 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:57.512288 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:57.513707 (+ 1419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:29:57.513715 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:29:57.513722 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:29:57.513747 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:29:57.513801 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:29:57.513819 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:29:57.513955 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:57.514126 (+ 171us) server_negotiation.cc:299] Negotiation successful
0806 09:29:57.514194 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260806 09:29:57.515532 3327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:57.508837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:42643 (local address 127.20.113.2:41581)
0806 09:29:57.509908 (+ 1071us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:57.509954 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:57.511821 (+ 1867us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:57.512572 (+ 751us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:57.512580 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:57.512667 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:29:57.513401 (+ 734us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 09:29:57.513403 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:29:57.513442 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 09:29:57.513527 (+ 85us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:29:57.513931 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:29:57.513940 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:29:57.513946 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 09:29:57.514256 (+ 310us) client_negotiation.cc:240] Negotiation successful
0806 09:29:57.514491 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":515,"thread_start_us":336,"threads_started":1}
I20260806 09:29:57.516942 3323 heartbeater.cc:347] Connected to a master server at 127.20.113.62:42643
I20260806 09:29:57.517252 3323 heartbeater.cc:464] Registering TS with master...
I20260806 09:29:57.517829 3323 heartbeater.cc:511] Master 127.20.113.62:42643 requested a full tablet report, sending...
I20260806 09:29:57.519439 2861 ts_manager.cc:194] Registered new tserver with Master: 640bf4f13e174bcfb2a3d239ab4adc8a (127.20.113.2:45449)
W20260806 09:29:57.864217 3328 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:29:57.864557 3328 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:29:57.864722 3328 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:29:57.871665 3328 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:29:57.871800 3328 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:29:57.872015 3328 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.113.3
I20260806 09:29:57.880967 3328 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:46749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786008495763054-20932-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786008495763054-20932-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.113.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786008495763054-20932-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.113.3
--webserver_port=0
--tserver_master_addrs=127.20.113.62:42643
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.113.3
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786008495763054-20932-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:29:57.882339 3328 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:29:57.884155 3328 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:29:57.897531 3338 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:29:57.897576 3336 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:29:57.897604 3335 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:29:57.898299 3328 server_base.cc:1034] running on GCE node
I20260806 09:29:57.898852 3328 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:29:57.899748 3328 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:29:57.900969 3328 hybrid_clock.cc:648] HybridClock initialized: now 1786008597900923 us; error 29 us; skew 500 ppm
I20260806 09:29:57.901295 3328 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 09:29:57.903636 3328 webserver.cc:460] Webserver started at http://127.20.113.3:35051/ using document root <none> and password file <none>
I20260806 09:29:57.904422 3328 fs_manager.cc:359] Metadata directory not provided
I20260806 09:29:57.904572 3328 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:29:57.904917 3328 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:29:57.907780 3328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786008495763054-20932-0/minicluster-data/ts-2/data/instance:
uuid: "d66a9be5afe84688b14df66d7d63d24a"
format_stamp: "Formatted at 2026-08-06 09:29:57 on dist-test-slave-45dx"
server_key: "f8a22c30a4848a43fc8dc877e766012a"
server_key_iv: "13f640a49c6893f27f76c8423163dacd"
server_key_version: "encryptionkey@0"
I20260806 09:29:57.908653 3328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786008495763054-20932-0/minicluster-data/ts-2/wal/instance:
uuid: "d66a9be5afe84688b14df66d7d63d24a"
format_stamp: "Formatted at 2026-08-06 09:29:57 on dist-test-slave-45dx"
server_key: "f8a22c30a4848a43fc8dc877e766012a"
server_key_iv: "13f640a49c6893f27f76c8423163dacd"
server_key_version: "encryptionkey@0"
I20260806 09:29:57.913811 3328 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260806 09:29:57.917611 3350 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:29:57.919181 3328 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 09:29:57.919345 3328 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786008495763054-20932-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786008495763054-20932-0/minicluster-data/ts-2/wal
uuid: "d66a9be5afe84688b14df66d7d63d24a"
format_stamp: "Formatted at 2026-08-06 09:29:57 on dist-test-slave-45dx"
server_key: "f8a22c30a4848a43fc8dc877e766012a"
server_key_iv: "13f640a49c6893f27f76c8423163dacd"
server_key_version: "encryptionkey@0"
I20260806 09:29:57.919580 3328 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786008495763054-20932-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786008495763054-20932-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786008495763054-20932-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:29:57.925268 3328 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:29:57.926038 3328 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:29:57.926312 3328 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:29:57.927248 3328 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:29:57.928717 3328 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:29:57.928836 3328 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:29:57.928974 3328 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:29:57.929078 3328 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:29:57.963987 3328 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.3:42977
I20260806 09:29:57.964032 3500 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.3:42977 every 8 connection(s)
I20260806 09:29:57.965384 3328 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786008495763054-20932-0/minicluster-data/ts-2/data/info.pb
I20260806 09:29:57.973181 3129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:57.967968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.3:57313 (local address 127.20.113.62:42643)
0806 09:29:57.968198 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:57.968204 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:57.969581 (+ 1377us) server_negotiation.cc:407] Connection header received
0806 09:29:57.971180 (+ 1599us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:57.971183 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:57.971237 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:57.971325 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:57.972571 (+ 1246us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:29:57.972575 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:29:57.972578 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:29:57.972593 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:29:57.972624 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:29:57.972635 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:29:57.972728 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:57.972892 (+ 164us) server_negotiation.cc:299] Negotiation successful
0806 09:29:57.972941 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260806 09:29:57.974056 3505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:57.968540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:42643 (local address 127.20.113.3:57313)
0806 09:29:57.969432 (+ 892us) negotiation.cc:107] Waiting for socket to connect
0806 09:29:57.969469 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 09:29:57.970953 (+ 1484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:29:57.971521 (+ 568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:29:57.971528 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:29:57.971624 (+ 96us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:29:57.972233 (+ 609us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 09:29:57.972235 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:29:57.972271 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 09:29:57.972413 (+ 142us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:29:57.972721 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:29:57.972729 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:29:57.972734 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 09:29:57.972944 (+ 210us) client_negotiation.cc:240] Negotiation successful
0806 09:29:57.973112 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":271,"threads_started":1}
I20260806 09:29:57.974769 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 3328
I20260806 09:29:57.975001 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786008495763054-20932-0/minicluster-data/ts-2/wal/instance
I20260806 09:29:57.975474 20932 external_mini_cluster.cc:1428] Setting key d288061a8eaea069d6a7e25dcd4c2b00
I20260806 09:29:57.975656 3502 heartbeater.cc:347] Connected to a master server at 127.20.113.62:42643
I20260806 09:29:57.975979 3502 heartbeater.cc:464] Registering TS with master...
I20260806 09:29:57.976696 3502 heartbeater.cc:511] Master 127.20.113.62:42643 requested a full tablet report, sending...
I20260806 09:29:57.978384 2859 ts_manager.cc:194] Registered new tserver with Master: d66a9be5afe84688b14df66d7d63d24a (127.20.113.3:42977)
I20260806 09:29:57.990859 20932 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 09:29:57.997273 3129 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 09:29:57.995506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55450 (local address 127.20.113.62:42643)
0806 09:29:57.995750 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:57.995757 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:57.996185 (+ 428us) server_negotiation.cc:407] Connection header received
0806 09:29:57.996421 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:57.996426 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:57.996491 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:57.996599 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:57.996997 (+ 398us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55450: BlockingRecv error: recv got EOF from 127.0.0.1:55450 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 09:29:57.997906 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 2945
I20260806 09:29:58.001516 3515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:57.998975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48868 (local address 127.20.113.1:35727)
0806 09:29:57.999528 (+ 553us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:57.999534 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:57.999559 (+ 25us) server_negotiation.cc:407] Connection header received
0806 09:29:57.999615 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:57.999620 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:57.999804 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:57.999929 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:58.000877 (+ 948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:29:58.000883 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:29:58.000896 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:29:58.000908 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:29:58.000951 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:29:58.000965 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:29:58.001069 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:58.001185 (+ 116us) server_negotiation.cc:299] Negotiation successful
0806 09:29:58.001261 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":394,"thread_start_us":275,"threads_started":1}
I20260806 09:29:58.002701 3118 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:29:58.082173 3127 heartbeater.cc:503] Master 127.20.113.62:42643 was elected leader, sending a full tablet report...
I20260806 09:29:58.083500 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 2945
W20260806 09:29:58.109256 2842 connection.cc:535] server connection from 127.20.113.1:46147 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 09:29:58.109447 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 3131
I20260806 09:29:58.113430 3517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:58.110507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44036 (local address 127.20.113.2:45449)
0806 09:29:58.111134 (+ 627us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:58.111148 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:58.111176 (+ 28us) server_negotiation.cc:407] Connection header received
0806 09:29:58.111241 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:58.111247 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:58.111475 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:58.111626 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:58.112653 (+ 1027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:29:58.112660 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:29:58.112672 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:29:58.112690 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:29:58.112731 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:29:58.112744 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:29:58.112854 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:58.112995 (+ 141us) server_negotiation.cc:299] Negotiation successful
0806 09:29:58.113129 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":463,"thread_start_us":325,"threads_started":1}
I20260806 09:29:58.114708 3314 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:29:58.194840 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 3131
I20260806 09:29:58.223583 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 3328
I20260806 09:29:58.227910 3521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:29:58.224699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45248 (local address 127.20.113.3:42977)
0806 09:29:58.225381 (+ 682us) server_negotiation.cc:206] Beginning negotiation
0806 09:29:58.225397 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 09:29:58.225429 (+ 32us) server_negotiation.cc:407] Connection header received
0806 09:29:58.225493 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:29:58.225499 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:29:58.225719 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:29:58.225911 (+ 192us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:29:58.227105 (+ 1194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:29:58.227111 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:29:58.227127 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:29:58.227144 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:29:58.227181 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:29:58.227194 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:29:58.227385 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0806 09:29:58.227491 (+ 106us) server_negotiation.cc:299] Negotiation successful
0806 09:29:58.227594 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":524,"thread_start_us":354,"threads_started":1}
I20260806 09:29:58.229351 3496 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:29:58.314972 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 3328
I20260806 09:29:58.339885 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 2814
I20260806 09:29:58.341207 2904 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:29:58.445407 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 2814
2026-08-06T09:29:58Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2500 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786008495763054-20932-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 09:29:58 dist-test-slave-45dx krb5kdc[3527](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 09:29:58 dist-test-slave-45dx krb5kdc[3527](info): set up 2 sockets
Aug 06 09:29:58 dist-test-slave-45dx krb5kdc[3527](info): commencing operation
krb5kdc: starting...
W20260806 09:29:59.752400 20932 mini_kdc.cc:121] Time spent starting KDC: real 1.257s user 0.005s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786008495763054-20932-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786008495763054-20932-0/krb5kdc/test-user.keytab.
Aug 06 09:29:59 dist-test-slave-45dx krb5kdc[3527](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008599, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T09:29:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T09:29:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.113.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.113.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.113.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab.
Entry for principal kudu/127.20.113.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab.
I20260806 09:30:00.076122 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786008495763054-20932-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786008495763054-20932-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786008495763054-20932-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.113.62:40793
--webserver_interface=127.20.113.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab
--principal=kudu/127.20.113.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:37049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.113.62:40793
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:30:00.426775 3546 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:30:00.427065 3546 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:30:00.427255 3546 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:30:00.434314 3546 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 09:30:00.434412 3546 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:30:00.434545 3546 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:30:00.434650 3546 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 09:30:00.434733 3546 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 09:30:00.444388 3546 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:37049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786008495763054-20932-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786008495763054-20932-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.113.62:40793
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab
--principal=kudu/127.20.113.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.113.62:40793
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.113.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786008495763054-20932-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:30:00.445685 3546 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:30:00.447019 3546 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:30:00.459403 3552 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:30:00.459460 3554 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:30:00.459362 3556 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:30:00.460206 3546 server_base.cc:1034] running on GCE node
I20260806 09:30:00.461071 3546 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:30:00.462590 3546 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:30:00.463837 3546 hybrid_clock.cc:648] HybridClock initialized: now 1786008600463793 us; error 56 us; skew 500 ppm
I20260806 09:30:00.464196 3546 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:30:00 dist-test-slave-45dx krb5kdc[3527](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008600, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:30:00.471918 3546 init.cc:377] Logged in from keytab as kudu/127.20.113.62@KRBTEST.COM (short username kudu)
I20260806 09:30:00.473511 3546 webserver.cc:460] Webserver started at http://127.20.113.62:43215/ using document root <none> and password file <none>
I20260806 09:30:00.474231 3546 fs_manager.cc:359] Metadata directory not provided
I20260806 09:30:00.474391 3546 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:30:00.474732 3546 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:30:00.477839 3546 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786008495763054-20932-0/minicluster-data/master-0/data/instance:
uuid: "7c897a4b87744bfb928a75bda9836791"
format_stamp: "Formatted at 2026-08-06 09:30:00 on dist-test-slave-45dx"
server_key: "a269583c3db679b7104ddb1e3496adaa"
server_key_iv: "126dc9e874e2405e81fb0631089c076f"
server_key_version: "encryptionkey@0"
I20260806 09:30:00.478643 3546 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786008495763054-20932-0/minicluster-data/master-0/wal/instance:
uuid: "7c897a4b87744bfb928a75bda9836791"
format_stamp: "Formatted at 2026-08-06 09:30:00 on dist-test-slave-45dx"
server_key: "a269583c3db679b7104ddb1e3496adaa"
server_key_iv: "126dc9e874e2405e81fb0631089c076f"
server_key_version: "encryptionkey@0"
I20260806 09:30:00.483907 3546 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260806 09:30:00.487340 3566 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:30:00.488806 3546 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 09:30:00.488988 3546 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786008495763054-20932-0/minicluster-data/master-0/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786008495763054-20932-0/minicluster-data/master-0/wal
uuid: "7c897a4b87744bfb928a75bda9836791"
format_stamp: "Formatted at 2026-08-06 09:30:00 on dist-test-slave-45dx"
server_key: "a269583c3db679b7104ddb1e3496adaa"
server_key_iv: "126dc9e874e2405e81fb0631089c076f"
server_key_version: "encryptionkey@0"
I20260806 09:30:00.489183 3546 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786008495763054-20932-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786008495763054-20932-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786008495763054-20932-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:30:00.507174 3546 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:30:00.511199 3546 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:30:00.511510 3546 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:30:00.529749 3546 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.62:40793
I20260806 09:30:00.529781 3646 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.62:40793 every 8 connection(s)
I20260806 09:30:00.531152 3546 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
I20260806 09:30:00.534943 3647 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:30:00.538918 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 3546
I20260806 09:30:00.539158 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786008495763054-20932-0/minicluster-data/master-0/wal/instance
I20260806 09:30:00.539831 20932 external_mini_cluster.cc:1428] Setting key 88437216179c539d3a67f1341ebc8780
I20260806 09:30:00.547648 3647 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c897a4b87744bfb928a75bda9836791: Bootstrap starting.
I20260806 09:30:00.551494 3647 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7c897a4b87744bfb928a75bda9836791: Neither blocks nor log segments found. Creating new log.
I20260806 09:30:00.552968 3647 log.cc:824] T 00000000000000000000000000000000 P 7c897a4b87744bfb928a75bda9836791: Log is configured to *not* fsync() on all Append() calls
I20260806 09:30:00.555809 3647 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c897a4b87744bfb928a75bda9836791: No bootstrap required, opened a new log
Aug 06 09:30:00 dist-test-slave-45dx krb5kdc[3527](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008599, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.113.62@KRBTEST.COM
I20260806 09:30:00.565673 3650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:30:00.541994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34148 (local address 127.20.113.62:40793)
0806 09:30:00.542816 (+ 822us) server_negotiation.cc:206] Beginning negotiation
0806 09:30:00.542837 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 09:30:00.542893 (+ 56us) server_negotiation.cc:407] Connection header received
0806 09:30:00.545839 (+ 2946us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:30:00.545869 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:30:00.546361 (+ 492us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:30:00.546815 (+ 454us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:30:00.548295 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:30:00.550374 (+ 2079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:30:00.551723 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:30:00.552877 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:30:00.553088 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:30:00.557104 (+ 4016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:30:00.557114 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:30:00.557133 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:30:00.557180 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:30:00.561168 (+ 3988us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:30:00.561849 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:30:00.561859 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:30:00.561867 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:30:00.562092 (+ 225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:30:00.562485 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:30:00.562489 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:30:00.562492 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:30:00.563206 (+ 714us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:30:00.563532 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0806 09:30:00.563834 (+ 302us) server_negotiation.cc:299] Negotiation successful
0806 09:30:00.564253 (+ 419us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":582,"thread_start_us":333,"threads_started":1}
I20260806 09:30:00.566855 3647 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7c897a4b87744bfb928a75bda9836791 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c897a4b87744bfb928a75bda9836791" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 40793 } }
I20260806 09:30:00.567247 3647 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7c897a4b87744bfb928a75bda9836791 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:30:00.567377 3647 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7c897a4b87744bfb928a75bda9836791 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c897a4b87744bfb928a75bda9836791, State: Initialized, Role: FOLLOWER
I20260806 09:30:00.568073 3647 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7c897a4b87744bfb928a75bda9836791 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c897a4b87744bfb928a75bda9836791" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 40793 } }
I20260806 09:30:00.568363 3647 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7c897a4b87744bfb928a75bda9836791 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 09:30:00.568548 3647 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7c897a4b87744bfb928a75bda9836791 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 09:30:00.568742 3647 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7c897a4b87744bfb928a75bda9836791 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:30:00.571381 3647 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7c897a4b87744bfb928a75bda9836791 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c897a4b87744bfb928a75bda9836791" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 40793 } }
I20260806 09:30:00.571820 3647 leader_election.cc:302] T 00000000000000000000000000000000 P 7c897a4b87744bfb928a75bda9836791 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7c897a4b87744bfb928a75bda9836791; no voters:
I20260806 09:30:00.572467 3647 leader_election.cc:288] T 00000000000000000000000000000000 P 7c897a4b87744bfb928a75bda9836791 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 09:30:00.572583 3652 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7c897a4b87744bfb928a75bda9836791 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:30:00.573360 3652 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7c897a4b87744bfb928a75bda9836791 [term 1 LEADER]: Becoming Leader. State: Replica: 7c897a4b87744bfb928a75bda9836791, State: Running, Role: LEADER
I20260806 09:30:00.573925 3652 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7c897a4b87744bfb928a75bda9836791 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c897a4b87744bfb928a75bda9836791" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 40793 } }
I20260806 09:30:00.574546 3647 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7c897a4b87744bfb928a75bda9836791 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 09:30:00.576798 3655 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c897a4b87744bfb928a75bda9836791 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7c897a4b87744bfb928a75bda9836791. Latest consensus state: current_term: 1 leader_uuid: "7c897a4b87744bfb928a75bda9836791" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c897a4b87744bfb928a75bda9836791" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 40793 } } }
I20260806 09:30:00.577149 3655 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c897a4b87744bfb928a75bda9836791 [sys.catalog]: This master's current role is: LEADER
I20260806 09:30:00.577682 3654 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c897a4b87744bfb928a75bda9836791 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7c897a4b87744bfb928a75bda9836791" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c897a4b87744bfb928a75bda9836791" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 40793 } } }
I20260806 09:30:00.578079 3654 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c897a4b87744bfb928a75bda9836791 [sys.catalog]: This master's current role is: LEADER
I20260806 09:30:00.578914 3658 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 09:30:00.583549 3658 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 09:30:00.594034 3658 catalog_manager.cc:1347] Generated new cluster ID: e42dd4c99d8b4e64b6e110852fca0c54
I20260806 09:30:00.594177 3658 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 09:30:00.603506 3658 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 09:30:00.604826 3658 catalog_manager.cc:1504] Loading token signing keys...
I20260806 09:30:00.611971 3658 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7c897a4b87744bfb928a75bda9836791: Generated new TSK 0
I20260806 09:30:00.612969 3658 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.113.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.113.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.113.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.1.keytab.
Entry for principal kudu/127.20.113.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.1.keytab.
I20260806 09:30:00.692076 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786008495763054-20932-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786008495763054-20932-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786008495763054-20932-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786008495763054-20932-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.113.1:0
--local_ip_for_outbound_sockets=127.20.113.1
--webserver_interface=127.20.113.1
--webserver_port=0
--tserver_master_addrs=127.20.113.62:40793
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.1.keytab
--principal=kudu/127.20.113.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:37049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:30:01.062734 3678 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:30:01.063148 3678 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:30:01.063328 3678 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:30:01.072098 3678 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:30:01.072203 3678 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:30:01.072422 3678 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.113.1
I20260806 09:30:01.082742 3678 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:37049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786008495763054-20932-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786008495763054-20932-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.1.keytab
--principal=kudu/127.20.113.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.113.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786008495763054-20932-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.113.1
--webserver_port=0
--tserver_master_addrs=127.20.113.62:40793
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.113.1
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786008495763054-20932-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:30:01.084102 3678 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:30:01.085564 3678 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:30:01.099469 3691 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:30:01.099530 3687 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:30:01.099576 3686 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:30:01.100541 3678 server_base.cc:1034] running on GCE node
I20260806 09:30:01.101164 3678 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:30:01.102160 3678 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:30:01.103404 3678 hybrid_clock.cc:648] HybridClock initialized: now 1786008601103366 us; error 54 us; skew 500 ppm
I20260806 09:30:01.103734 3678 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:30:01 dist-test-slave-45dx krb5kdc[3527](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008601, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:30:01.107215 3678 init.cc:377] Logged in from keytab as kudu/127.20.113.1@KRBTEST.COM (short username kudu)
I20260806 09:30:01.108752 3678 webserver.cc:460] Webserver started at http://127.20.113.1:45389/ using document root <none> and password file <none>
I20260806 09:30:01.109483 3678 fs_manager.cc:359] Metadata directory not provided
I20260806 09:30:01.109640 3678 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:30:01.109969 3678 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:30:01.113183 3678 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786008495763054-20932-0/minicluster-data/ts-0/data/instance:
uuid: "c86cc3847e78481d84faace4cc1fddba"
format_stamp: "Formatted at 2026-08-06 09:30:01 on dist-test-slave-45dx"
server_key: "f2943663a59bf21bd3742da2597c555d"
server_key_iv: "b4c93e4144a67f9382514c593cf8ae4a"
server_key_version: "encryptionkey@0"
I20260806 09:30:01.114017 3678 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786008495763054-20932-0/minicluster-data/ts-0/wal/instance:
uuid: "c86cc3847e78481d84faace4cc1fddba"
format_stamp: "Formatted at 2026-08-06 09:30:01 on dist-test-slave-45dx"
server_key: "f2943663a59bf21bd3742da2597c555d"
server_key_iv: "b4c93e4144a67f9382514c593cf8ae4a"
server_key_version: "encryptionkey@0"
I20260806 09:30:01.119722 3678 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260806 09:30:01.123386 3700 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:30:01.124892 3678 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 09:30:01.125082 3678 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786008495763054-20932-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786008495763054-20932-0/minicluster-data/ts-0/wal
uuid: "c86cc3847e78481d84faace4cc1fddba"
format_stamp: "Formatted at 2026-08-06 09:30:01 on dist-test-slave-45dx"
server_key: "f2943663a59bf21bd3742da2597c555d"
server_key_iv: "b4c93e4144a67f9382514c593cf8ae4a"
server_key_version: "encryptionkey@0"
I20260806 09:30:01.125291 3678 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786008495763054-20932-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786008495763054-20932-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786008495763054-20932-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:30:01.138695 3678 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:30:01.143067 3678 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:30:01.143399 3678 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:30:01.144475 3678 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:30:01.145896 3678 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:30:01.146015 3678 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:30:01.146176 3678 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:30:01.146263 3678 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:30:01.183022 3678 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.1:34241
I20260806 09:30:01.183071 3854 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.1:34241 every 8 connection(s)
I20260806 09:30:01.184496 3678 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786008495763054-20932-0/minicluster-data/ts-0/data/info.pb
I20260806 09:30:01.190009 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 3678
I20260806 09:30:01.190258 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786008495763054-20932-0/minicluster-data/ts-0/wal/instance
I20260806 09:30:01.191107 20932 external_mini_cluster.cc:1428] Setting key d8be1c498fb1d831f95e078873567f77
Aug 06 09:30:01 dist-test-slave-45dx krb5kdc[3527](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008601, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.1@KRBTEST.COM for kudu/127.20.113.62@KRBTEST.COM
I20260806 09:30:01.205729 3858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:30:01.187324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.1:40441 (local address 127.20.113.62:40793)
0806 09:30:01.187841 (+ 517us) server_negotiation.cc:206] Beginning negotiation
0806 09:30:01.187845 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 09:30:01.188788 (+ 943us) server_negotiation.cc:407] Connection header received
0806 09:30:01.191365 (+ 2577us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:30:01.191370 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:30:01.191449 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:30:01.191554 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:30:01.193318 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:30:01.194053 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:30:01.195663 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:30:01.196072 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:30:01.196224 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:30:01.199787 (+ 3563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:30:01.199792 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:30:01.199795 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:30:01.199828 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:30:01.202546 (+ 2718us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:30:01.203313 (+ 767us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:30:01.203323 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:30:01.203326 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:30:01.203455 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:30:01.203914 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:30:01.203923 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:30:01.203926 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:30:01.204191 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:30:01.204376 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0806 09:30:01.205061 (+ 685us) server_negotiation.cc:299] Negotiation successful
0806 09:30:01.205353 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":415,"thread_start_us":281,"threads_started":1}
I20260806 09:30:01.206715 3859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:30:01.187818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:40793 (local address 127.20.113.1:40441)
0806 09:30:01.188615 (+ 797us) negotiation.cc:107] Waiting for socket to connect
0806 09:30:01.188663 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 09:30:01.190999 (+ 2336us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:30:01.191852 (+ 853us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:30:01.191861 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:30:01.192479 (+ 618us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:30:01.193138 (+ 659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:30:01.193159 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:30:01.194232 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:30:01.194237 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:30:01.195489 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:30:01.195502 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:30:01.196342 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:30:01.196352 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:30:01.196583 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:30:01.197291 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:30:01.197313 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:30:01.199550 (+ 2237us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:30:01.202716 (+ 3166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:30:01.202725 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:30:01.202741 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:30:01.203176 (+ 435us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:30:01.203575 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:30:01.203581 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:30:01.203584 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:30:01.203784 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:30:01.204347 (+ 563us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:30:01.204355 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:30:01.204684 (+ 329us) client_negotiation.cc:769] Sending connection context
0806 09:30:01.204952 (+ 268us) client_negotiation.cc:240] Negotiation successful
0806 09:30:01.205530 (+ 578us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":566,"thread_start_us":292,"threads_started":1}
I20260806 09:30:01.208585 3856 heartbeater.cc:347] Connected to a master server at 127.20.113.62:40793
I20260806 09:30:01.208964 3856 heartbeater.cc:464] Registering TS with master...
I20260806 09:30:01.209986 3856 heartbeater.cc:511] Master 127.20.113.62:40793 requested a full tablet report, sending...
I20260806 09:30:01.212608 3587 ts_manager.cc:194] Registered new tserver with Master: c86cc3847e78481d84faace4cc1fddba (127.20.113.1:34241)
I20260806 09:30:01.215126 3587 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.113.1@KRBTEST.COM'} at 127.20.113.1:40441
WARNING: no policy specified for kudu/127.20.113.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.113.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.113.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.2.keytab.
Entry for principal kudu/127.20.113.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.2.keytab.
I20260806 09:30:01.271552 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786008495763054-20932-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786008495763054-20932-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786008495763054-20932-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786008495763054-20932-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.113.2:0
--local_ip_for_outbound_sockets=127.20.113.2
--webserver_interface=127.20.113.2
--webserver_port=0
--tserver_master_addrs=127.20.113.62:40793
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.2.keytab
--principal=kudu/127.20.113.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:37049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:30:01.629393 3865 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:30:01.629676 3865 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:30:01.629833 3865 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:30:01.637176 3865 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:30:01.637322 3865 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:30:01.637457 3865 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.113.2
I20260806 09:30:01.648499 3865 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:37049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786008495763054-20932-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786008495763054-20932-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.2.keytab
--principal=kudu/127.20.113.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.113.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786008495763054-20932-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.113.2
--webserver_port=0
--tserver_master_addrs=127.20.113.62:40793
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.113.2
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786008495763054-20932-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:30:01.649767 3865 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:30:01.651214 3865 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:30:01.664620 3874 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:30:01.664577 3870 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:30:01.664672 3871 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:30:01.669950 3865 server_base.cc:1034] running on GCE node
I20260806 09:30:01.670708 3865 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:30:01.671772 3865 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:30:01.673026 3865 hybrid_clock.cc:648] HybridClock initialized: now 1786008601672956 us; error 60 us; skew 500 ppm
I20260806 09:30:01.673377 3865 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:30:01 dist-test-slave-45dx krb5kdc[3527](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008601, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:30:01.676860 3865 init.cc:377] Logged in from keytab as kudu/127.20.113.2@KRBTEST.COM (short username kudu)
I20260806 09:30:01.678386 3865 webserver.cc:460] Webserver started at http://127.20.113.2:35267/ using document root <none> and password file <none>
I20260806 09:30:01.679113 3865 fs_manager.cc:359] Metadata directory not provided
I20260806 09:30:01.679217 3865 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:30:01.679471 3865 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:30:01.682482 3865 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786008495763054-20932-0/minicluster-data/ts-1/data/instance:
uuid: "419cd963fbdb4d7fbb7f8b16833bfb18"
format_stamp: "Formatted at 2026-08-06 09:30:01 on dist-test-slave-45dx"
server_key: "028016a2b2286c9c9e424ebf109a45b1"
server_key_iv: "5faef4634b9a4130ee5d6409e1dd5069"
server_key_version: "encryptionkey@0"
I20260806 09:30:01.683208 3865 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786008495763054-20932-0/minicluster-data/ts-1/wal/instance:
uuid: "419cd963fbdb4d7fbb7f8b16833bfb18"
format_stamp: "Formatted at 2026-08-06 09:30:01 on dist-test-slave-45dx"
server_key: "028016a2b2286c9c9e424ebf109a45b1"
server_key_iv: "5faef4634b9a4130ee5d6409e1dd5069"
server_key_version: "encryptionkey@0"
I20260806 09:30:01.688846 3865 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260806 09:30:01.692358 3885 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:30:01.693765 3865 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 09:30:01.693905 3865 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786008495763054-20932-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786008495763054-20932-0/minicluster-data/ts-1/wal
uuid: "419cd963fbdb4d7fbb7f8b16833bfb18"
format_stamp: "Formatted at 2026-08-06 09:30:01 on dist-test-slave-45dx"
server_key: "028016a2b2286c9c9e424ebf109a45b1"
server_key_iv: "5faef4634b9a4130ee5d6409e1dd5069"
server_key_version: "encryptionkey@0"
I20260806 09:30:01.694080 3865 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786008495763054-20932-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786008495763054-20932-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786008495763054-20932-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:30:01.726300 3865 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:30:01.730769 3865 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:30:01.731087 3865 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:30:01.732102 3865 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:30:01.733534 3865 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:30:01.733613 3865 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:30:01.733779 3865 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:30:01.733832 3865 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:30:01.768208 3865 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.2:39119
I20260806 09:30:01.768267 4040 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.2:39119 every 8 connection(s)
I20260806 09:30:01.769634 3865 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786008495763054-20932-0/minicluster-data/ts-1/data/info.pb
I20260806 09:30:01.773267 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 3865
I20260806 09:30:01.773476 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786008495763054-20932-0/minicluster-data/ts-1/wal/instance
I20260806 09:30:01.774019 20932 external_mini_cluster.cc:1428] Setting key 28aa3c88980246b6b46864953ab06f9b
Aug 06 09:30:01 dist-test-slave-45dx krb5kdc[3527](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008601, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.2@KRBTEST.COM for kudu/127.20.113.62@KRBTEST.COM
I20260806 09:30:01.794031 4043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:30:01.772203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.2:40615 (local address 127.20.113.62:40793)
0806 09:30:01.772731 (+ 528us) server_negotiation.cc:206] Beginning negotiation
0806 09:30:01.772736 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:30:01.773840 (+ 1104us) server_negotiation.cc:407] Connection header received
0806 09:30:01.776058 (+ 2218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:30:01.776064 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:30:01.776147 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:30:01.776253 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:30:01.778651 (+ 2398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:30:01.779400 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:30:01.782090 (+ 2690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:30:01.782511 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:30:01.782607 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:30:01.786967 (+ 4360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:30:01.786977 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:30:01.786984 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:30:01.787031 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:30:01.790697 (+ 3666us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:30:01.791428 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:30:01.791436 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:30:01.791442 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:30:01.791613 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:30:01.792081 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:30:01.792086 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:30:01.792088 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:30:01.792443 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:30:01.792604 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0806 09:30:01.793366 (+ 762us) server_negotiation.cc:299] Negotiation successful
0806 09:30:01.793612 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":427,"thread_start_us":306,"threads_started":1}
I20260806 09:30:01.795081 4044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:30:01.772741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:40793 (local address 127.20.113.2:40615)
0806 09:30:01.773658 (+ 917us) negotiation.cc:107] Waiting for socket to connect
0806 09:30:01.773698 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 09:30:01.775749 (+ 2051us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:30:01.776601 (+ 852us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:30:01.776616 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:30:01.777635 (+ 1019us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:30:01.778455 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:30:01.778484 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:30:01.779572 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:30:01.779580 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:30:01.781928 (+ 2348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:30:01.781948 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:30:01.782754 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:30:01.782764 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:30:01.783100 (+ 336us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:30:01.783869 (+ 769us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:30:01.783917 (+ 48us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:30:01.786714 (+ 2797us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:30:01.790840 (+ 4126us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:30:01.790846 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:30:01.790858 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:30:01.791292 (+ 434us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:30:01.791724 (+ 432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:30:01.791727 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:30:01.791729 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:30:01.791925 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:30:01.792577 (+ 652us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:30:01.792584 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:30:01.792947 (+ 363us) client_negotiation.cc:769] Sending connection context
0806 09:30:01.793258 (+ 311us) client_negotiation.cc:240] Negotiation successful
0806 09:30:01.793823 (+ 565us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":622,"thread_start_us":346,"threads_started":1}
I20260806 09:30:01.796679 4041 heartbeater.cc:347] Connected to a master server at 127.20.113.62:40793
I20260806 09:30:01.797019 4041 heartbeater.cc:464] Registering TS with master...
I20260806 09:30:01.797912 4041 heartbeater.cc:511] Master 127.20.113.62:40793 requested a full tablet report, sending...
I20260806 09:30:01.799500 3587 ts_manager.cc:194] Registered new tserver with Master: 419cd963fbdb4d7fbb7f8b16833bfb18 (127.20.113.2:39119)
I20260806 09:30:01.800392 3587 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.113.2@KRBTEST.COM'} at 127.20.113.2:40615
WARNING: no policy specified for kudu/127.20.113.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.113.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.113.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.3.keytab.
Entry for principal kudu/127.20.113.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.3.keytab.
I20260806 09:30:01.850976 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786008495763054-20932-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786008495763054-20932-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786008495763054-20932-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786008495763054-20932-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.113.3:0
--local_ip_for_outbound_sockets=127.20.113.3
--webserver_interface=127.20.113.3
--webserver_port=0
--tserver_master_addrs=127.20.113.62:40793
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.3.keytab
--principal=kudu/127.20.113.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:37049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:30:02.197193 4047 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:30:02.197544 4047 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:30:02.197661 4047 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:30:02.204962 4047 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:30:02.205068 4047 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:30:02.205180 4047 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.113.3
I20260806 09:30:02.214804 4047 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:37049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786008495763054-20932-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786008495763054-20932-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.3.keytab
--principal=kudu/127.20.113.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.113.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786008495763054-20932-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.113.3
--webserver_port=0
--tserver_master_addrs=127.20.113.62:40793
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.113.3
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786008495763054-20932-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:30:02.216077 4047 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:30:02.217407 4047 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 09:30:02.219309 3856 heartbeater.cc:503] Master 127.20.113.62:40793 was elected leader, sending a full tablet report...
W20260806 09:30:02.231580 4056 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:30:02.231737 4060 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:30:02.231971 4062 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:30:02.233071 4047 server_base.cc:1034] running on GCE node
I20260806 09:30:02.233693 4047 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:30:02.234642 4047 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:30:02.235891 4047 hybrid_clock.cc:648] HybridClock initialized: now 1786008602235840 us; error 71 us; skew 500 ppm
I20260806 09:30:02.236222 4047 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:30:02 dist-test-slave-45dx krb5kdc[3527](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008602, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:30:02.239490 4047 init.cc:377] Logged in from keytab as kudu/127.20.113.3@KRBTEST.COM (short username kudu)
I20260806 09:30:02.240973 4047 webserver.cc:460] Webserver started at http://127.20.113.3:42359/ using document root <none> and password file <none>
I20260806 09:30:02.241678 4047 fs_manager.cc:359] Metadata directory not provided
I20260806 09:30:02.241863 4047 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:30:02.242174 4047 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:30:02.245071 4047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786008495763054-20932-0/minicluster-data/ts-2/data/instance:
uuid: "889f19c67adc423d8abddec2a680acaa"
format_stamp: "Formatted at 2026-08-06 09:30:02 on dist-test-slave-45dx"
server_key: "1a50a1c45299033489d3f96fdfc87cc8"
server_key_iv: "237dbe926554ec3e13cb26c757576343"
server_key_version: "encryptionkey@0"
I20260806 09:30:02.245834 4047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786008495763054-20932-0/minicluster-data/ts-2/wal/instance:
uuid: "889f19c67adc423d8abddec2a680acaa"
format_stamp: "Formatted at 2026-08-06 09:30:02 on dist-test-slave-45dx"
server_key: "1a50a1c45299033489d3f96fdfc87cc8"
server_key_iv: "237dbe926554ec3e13cb26c757576343"
server_key_version: "encryptionkey@0"
I20260806 09:30:02.250993 4047 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 09:30:02.254498 4078 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:30:02.255816 4047 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 09:30:02.256022 4047 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786008495763054-20932-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786008495763054-20932-0/minicluster-data/ts-2/wal
uuid: "889f19c67adc423d8abddec2a680acaa"
format_stamp: "Formatted at 2026-08-06 09:30:02 on dist-test-slave-45dx"
server_key: "1a50a1c45299033489d3f96fdfc87cc8"
server_key_iv: "237dbe926554ec3e13cb26c757576343"
server_key_version: "encryptionkey@0"
I20260806 09:30:02.256240 4047 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786008495763054-20932-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786008495763054-20932-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786008495763054-20932-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:30:02.299116 4047 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:30:02.303643 4047 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:30:02.303980 4047 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:30:02.305032 4047 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:30:02.306411 4047 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:30:02.306536 4047 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:30:02.306689 4047 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:30:02.306767 4047 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:30:02.342165 4047 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.3:43663
I20260806 09:30:02.342228 4232 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.3:43663 every 8 connection(s)
I20260806 09:30:02.343536 4047 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786008495763054-20932-0/minicluster-data/ts-2/data/info.pb
I20260806 09:30:02.351953 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 4047
I20260806 09:30:02.352192 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786008495763054-20932-0/minicluster-data/ts-2/wal/instance
I20260806 09:30:02.352674 20932 external_mini_cluster.cc:1428] Setting key 307a8bee78b3291ea3f9d345f5e256e2
Aug 06 09:30:02 dist-test-slave-45dx krb5kdc[3527](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008602, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.3@KRBTEST.COM for kudu/127.20.113.62@KRBTEST.COM
I20260806 09:30:02.364473 4237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:30:02.346372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.3:57815 (local address 127.20.113.62:40793)
0806 09:30:02.346862 (+ 490us) server_negotiation.cc:206] Beginning negotiation
0806 09:30:02.346867 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:30:02.347879 (+ 1012us) server_negotiation.cc:407] Connection header received
0806 09:30:02.349814 (+ 1935us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:30:02.349820 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:30:02.349908 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:30:02.350035 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:30:02.351465 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:30:02.352393 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:30:02.354436 (+ 2043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:30:02.354849 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:30:02.354996 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:30:02.358689 (+ 3693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:30:02.358695 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:30:02.358700 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:30:02.358742 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:30:02.361353 (+ 2611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:30:02.362113 (+ 760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:30:02.362120 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:30:02.362123 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:30:02.362258 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:30:02.362712 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:30:02.362717 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:30:02.362721 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:30:02.362987 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:30:02.363168 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0806 09:30:02.363836 (+ 668us) server_negotiation.cc:299] Negotiation successful
0806 09:30:02.364054 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":241,"threads_started":1}
I20260806 09:30:02.365437 4238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:30:02.346984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:40793 (local address 127.20.113.3:57815)
0806 09:30:02.347701 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0806 09:30:02.347748 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 09:30:02.349550 (+ 1802us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:30:02.350261 (+ 711us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:30:02.350269 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:30:02.350749 (+ 480us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:30:02.351274 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:30:02.351290 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:30:02.352544 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:30:02.352551 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:30:02.354258 (+ 1707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:30:02.354270 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:30:02.355086 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:30:02.355093 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:30:02.355364 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:30:02.356086 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:30:02.356108 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:30:02.358477 (+ 2369us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:30:02.361490 (+ 3013us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:30:02.361500 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:30:02.361515 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:30:02.361986 (+ 471us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:30:02.362386 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:30:02.362391 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:30:02.362395 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:30:02.362591 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:30:02.363144 (+ 553us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:30:02.363152 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:30:02.363478 (+ 326us) client_negotiation.cc:769] Sending connection context
0806 09:30:02.363742 (+ 264us) client_negotiation.cc:240] Negotiation successful
0806 09:30:02.364259 (+ 517us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":286,"threads_started":1}
I20260806 09:30:02.367012 4234 heartbeater.cc:347] Connected to a master server at 127.20.113.62:40793
I20260806 09:30:02.367344 4234 heartbeater.cc:464] Registering TS with master...
I20260806 09:30:02.368098 4234 heartbeater.cc:511] Master 127.20.113.62:40793 requested a full tablet report, sending...
I20260806 09:30:02.369614 3587 ts_manager.cc:194] Registered new tserver with Master: 889f19c67adc423d8abddec2a680acaa (127.20.113.3:43663)
I20260806 09:30:02.370565 3587 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.113.3@KRBTEST.COM'} at 127.20.113.3:57815
I20260806 09:30:02.380499 20932 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 09:30:02.398087 4237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:30:02.385212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36542 (local address 127.20.113.62:40793)
0806 09:30:02.385435 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0806 09:30:02.385444 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 09:30:02.386036 (+ 592us) server_negotiation.cc:407] Connection header received
0806 09:30:02.386181 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:30:02.386184 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:30:02.386247 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:30:02.386326 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:30:02.387766 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:30:02.388577 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:30:02.389695 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:30:02.390063 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:30:02.390155 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:30:02.392499 (+ 2344us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:30:02.392504 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:30:02.392508 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:30:02.392540 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:30:02.395211 (+ 2671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:30:02.395887 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:30:02.395893 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:30:02.395897 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:30:02.396037 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:30:02.396510 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:30:02.396515 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:30:02.396521 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:30:02.396848 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:30:02.397114 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0806 09:30:02.397510 (+ 396us) server_negotiation.cc:299] Negotiation successful
0806 09:30:02.397765 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260806 09:30:02.402133 3587 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36542:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 09:30:02.409250 3587 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 09:30:02.443128 4258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:30:02.423796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47880 (local address 127.20.113.1:34241)
0806 09:30:02.433275 (+ 9479us) server_negotiation.cc:206] Beginning negotiation
0806 09:30:02.433286 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 09:30:02.434412 (+ 1126us) server_negotiation.cc:407] Connection header received
0806 09:30:02.434523 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:30:02.434531 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:30:02.434766 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:30:02.435410 (+ 644us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:30:02.436169 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:30:02.437735 (+ 1566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:30:02.440062 (+ 2327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:30:02.441657 (+ 1595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:30:02.441905 (+ 248us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:30:02.442175 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0806 09:30:02.442531 (+ 356us) server_negotiation.cc:299] Negotiation successful
0806 09:30:02.442742 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9284,"thread_start_us":9096,"threads_started":1}
I20260806 09:30:02.443279 4255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:30:02.423269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.1:34241 (local address 127.0.0.1:47880)
0806 09:30:02.434153 (+ 10884us) negotiation.cc:107] Waiting for socket to connect
0806 09:30:02.434169 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 09:30:02.434290 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:30:02.435040 (+ 750us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:30:02.435044 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:30:02.435078 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:30:02.435921 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:30:02.435949 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:30:02.437895 (+ 1946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:30:02.437899 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:30:02.439855 (+ 1956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:30:02.439871 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:30:02.441985 (+ 2114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:30:02.441991 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:30:02.442192 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:30:02.442246 (+ 54us) client_negotiation.cc:769] Sending connection context
0806 09:30:02.442403 (+ 157us) client_negotiation.cc:240] Negotiation successful
0806 09:30:02.442718 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10765,"mutex_wait_us":416,"spinlock_wait_cycles":4480,"thread_start_us":10609,"threads_started":1}
I20260806 09:30:02.443759 4252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:30:02.421901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.3:43663 (local address 127.0.0.1:34010)
0806 09:30:02.422703 (+ 802us) negotiation.cc:107] Waiting for socket to connect
0806 09:30:02.422794 (+ 91us) client_negotiation.cc:174] Beginning negotiation
0806 09:30:02.422989 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:30:02.434829 (+ 11840us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:30:02.434840 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:30:02.434885 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:30:02.435594 (+ 709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:30:02.435608 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:30:02.437991 (+ 2383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:30:02.438000 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:30:02.440574 (+ 2574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:30:02.440590 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:30:02.442808 (+ 2218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:30:02.442815 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:30:02.443026 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:30:02.443062 (+ 36us) client_negotiation.cc:769] Sending connection context
0806 09:30:02.443144 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 09:30:02.443306 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":509,"mutex_wait_us":70,"spinlock_wait_cycles":9088,"thread_start_us":352,"threads_started":1}
I20260806 09:30:02.444849 4254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:30:02.422694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.2:39119 (local address 127.0.0.1:51046)
0806 09:30:02.433135 (+ 10441us) negotiation.cc:107] Waiting for socket to connect
0806 09:30:02.433164 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0806 09:30:02.433307 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:30:02.436965 (+ 3658us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:30:02.436971 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:30:02.437001 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:30:02.437434 (+ 433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:30:02.437447 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:30:02.439826 (+ 2379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:30:02.439833 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:30:02.441650 (+ 1817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:30:02.441664 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:30:02.443924 (+ 2260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:30:02.443935 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:30:02.444162 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:30:02.444194 (+ 32us) client_negotiation.cc:769] Sending connection context
0806 09:30:02.444282 (+ 88us) client_negotiation.cc:240] Negotiation successful
0806 09:30:02.444444 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10290,"thread_start_us":10131,"threads_started":1}
I20260806 09:30:02.445295 4253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:30:02.422303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34010 (local address 127.20.113.3:43663)
0806 09:30:02.433807 (+ 11504us) server_negotiation.cc:206] Beginning negotiation
0806 09:30:02.433819 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 09:30:02.433867 (+ 48us) server_negotiation.cc:407] Connection header received
0806 09:30:02.433996 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:30:02.434008 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:30:02.434497 (+ 489us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:30:02.435359 (+ 862us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:30:02.435954 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:30:02.437802 (+ 1848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:30:02.440802 (+ 3000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:30:02.442542 (+ 1740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:30:02.444220 (+ 1678us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:30:02.444563 (+ 343us) server_negotiation.cc:1035] Waiting for connection context
0806 09:30:02.444641 (+ 78us) server_negotiation.cc:299] Negotiation successful
0806 09:30:02.444840 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11272,"thread_start_us":11131,"threads_started":1}
I20260806 09:30:02.446221 4256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:30:02.423339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51046 (local address 127.20.113.2:39119)
0806 09:30:02.436227 (+ 12888us) server_negotiation.cc:206] Beginning negotiation
0806 09:30:02.436237 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 09:30:02.436265 (+ 28us) server_negotiation.cc:407] Connection header received
0806 09:30:02.436401 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:30:02.436410 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:30:02.436649 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:30:02.436853 (+ 204us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:30:02.437878 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:30:02.439644 (+ 1766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:30:02.441905 (+ 2261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:30:02.443633 (+ 1728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:30:02.444553 (+ 920us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:30:02.445640 (+ 1087us) server_negotiation.cc:1035] Waiting for connection context
0806 09:30:02.445722 (+ 82us) server_negotiation.cc:299] Negotiation successful
0806 09:30:02.445900 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12713,"thread_start_us":12601,"threads_started":1}
I20260806 09:30:02.449503 3951 tablet_service.cc:1467] Processing CreateTablet for tablet 3afc962059534798867f4bc37841552f (DEFAULT_TABLE table=test-table [id=fbf8afa0663c47e3b1a181f327a1d48f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:30:02.449928 4143 tablet_service.cc:1467] Processing CreateTablet for tablet 3afc962059534798867f4bc37841552f (DEFAULT_TABLE table=test-table [id=fbf8afa0663c47e3b1a181f327a1d48f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:30:02.451246 3951 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3afc962059534798867f4bc37841552f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:30:02.451550 4143 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3afc962059534798867f4bc37841552f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:30:02.448076 3767 tablet_service.cc:1467] Processing CreateTablet for tablet 3afc962059534798867f4bc37841552f (DEFAULT_TABLE table=test-table [id=fbf8afa0663c47e3b1a181f327a1d48f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:30:02.455382 3767 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3afc962059534798867f4bc37841552f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:30:02.467726 4261 tablet_bootstrap.cc:492] T 3afc962059534798867f4bc37841552f P 889f19c67adc423d8abddec2a680acaa: Bootstrap starting.
I20260806 09:30:02.468740 4260 tablet_bootstrap.cc:492] T 3afc962059534798867f4bc37841552f P 419cd963fbdb4d7fbb7f8b16833bfb18: Bootstrap starting.
I20260806 09:30:02.471705 4261 tablet_bootstrap.cc:654] T 3afc962059534798867f4bc37841552f P 889f19c67adc423d8abddec2a680acaa: Neither blocks nor log segments found. Creating new log.
I20260806 09:30:02.472733 4260 tablet_bootstrap.cc:654] T 3afc962059534798867f4bc37841552f P 419cd963fbdb4d7fbb7f8b16833bfb18: Neither blocks nor log segments found. Creating new log.
I20260806 09:30:02.472749 4262 tablet_bootstrap.cc:492] T 3afc962059534798867f4bc37841552f P c86cc3847e78481d84faace4cc1fddba: Bootstrap starting.
I20260806 09:30:02.473215 4261 log.cc:824] T 3afc962059534798867f4bc37841552f P 889f19c67adc423d8abddec2a680acaa: Log is configured to *not* fsync() on all Append() calls
I20260806 09:30:02.474444 4260 log.cc:824] T 3afc962059534798867f4bc37841552f P 419cd963fbdb4d7fbb7f8b16833bfb18: Log is configured to *not* fsync() on all Append() calls
I20260806 09:30:02.476789 4262 tablet_bootstrap.cc:654] T 3afc962059534798867f4bc37841552f P c86cc3847e78481d84faace4cc1fddba: Neither blocks nor log segments found. Creating new log.
I20260806 09:30:02.477047 4261 tablet_bootstrap.cc:492] T 3afc962059534798867f4bc37841552f P 889f19c67adc423d8abddec2a680acaa: No bootstrap required, opened a new log
I20260806 09:30:02.477350 4261 ts_tablet_manager.cc:1397] T 3afc962059534798867f4bc37841552f P 889f19c67adc423d8abddec2a680acaa: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260806 09:30:02.478036 4262 log.cc:824] T 3afc962059534798867f4bc37841552f P c86cc3847e78481d84faace4cc1fddba: Log is configured to *not* fsync() on all Append() calls
I20260806 09:30:02.478507 4260 tablet_bootstrap.cc:492] T 3afc962059534798867f4bc37841552f P 419cd963fbdb4d7fbb7f8b16833bfb18: No bootstrap required, opened a new log
I20260806 09:30:02.478826 4260 ts_tablet_manager.cc:1397] T 3afc962059534798867f4bc37841552f P 419cd963fbdb4d7fbb7f8b16833bfb18: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.003s
I20260806 09:30:02.481344 4262 tablet_bootstrap.cc:492] T 3afc962059534798867f4bc37841552f P c86cc3847e78481d84faace4cc1fddba: No bootstrap required, opened a new log
I20260806 09:30:02.481670 4262 ts_tablet_manager.cc:1397] T 3afc962059534798867f4bc37841552f P c86cc3847e78481d84faace4cc1fddba: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260806 09:30:02.487174 4260 raft_consensus.cc:348] T 3afc962059534798867f4bc37841552f P 419cd963fbdb4d7fbb7f8b16833bfb18 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "889f19c67adc423d8abddec2a680acaa" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 43663 } } peers { permanent_uuid: "c86cc3847e78481d84faace4cc1fddba" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 34241 } } peers { permanent_uuid: "419cd963fbdb4d7fbb7f8b16833bfb18" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 39119 } }
I20260806 09:30:02.487604 4260 raft_consensus.cc:374] T 3afc962059534798867f4bc37841552f P 419cd963fbdb4d7fbb7f8b16833bfb18 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:30:02.487718 4260 raft_consensus.cc:729] T 3afc962059534798867f4bc37841552f P 419cd963fbdb4d7fbb7f8b16833bfb18 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 419cd963fbdb4d7fbb7f8b16833bfb18, State: Initialized, Role: FOLLOWER
I20260806 09:30:02.488471 4260 consensus_queue.cc:260] T 3afc962059534798867f4bc37841552f P 419cd963fbdb4d7fbb7f8b16833bfb18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "889f19c67adc423d8abddec2a680acaa" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 43663 } } peers { permanent_uuid: "c86cc3847e78481d84faace4cc1fddba" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 34241 } } peers { permanent_uuid: "419cd963fbdb4d7fbb7f8b16833bfb18" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 39119 } }
I20260806 09:30:02.488577 4261 raft_consensus.cc:348] T 3afc962059534798867f4bc37841552f P 889f19c67adc423d8abddec2a680acaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "889f19c67adc423d8abddec2a680acaa" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 43663 } } peers { permanent_uuid: "c86cc3847e78481d84faace4cc1fddba" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 34241 } } peers { permanent_uuid: "419cd963fbdb4d7fbb7f8b16833bfb18" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 39119 } }
I20260806 09:30:02.489033 4261 raft_consensus.cc:374] T 3afc962059534798867f4bc37841552f P 889f19c67adc423d8abddec2a680acaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:30:02.489140 4261 raft_consensus.cc:729] T 3afc962059534798867f4bc37841552f P 889f19c67adc423d8abddec2a680acaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 889f19c67adc423d8abddec2a680acaa, State: Initialized, Role: FOLLOWER
I20260806 09:30:02.489820 4041 heartbeater.cc:503] Master 127.20.113.62:40793 was elected leader, sending a full tablet report...
I20260806 09:30:02.489742 4261 consensus_queue.cc:260] T 3afc962059534798867f4bc37841552f P 889f19c67adc423d8abddec2a680acaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "889f19c67adc423d8abddec2a680acaa" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 43663 } } peers { permanent_uuid: "c86cc3847e78481d84faace4cc1fddba" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 34241 } } peers { permanent_uuid: "419cd963fbdb4d7fbb7f8b16833bfb18" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 39119 } }
I20260806 09:30:02.490607 4260 ts_tablet_manager.cc:1428] T 3afc962059534798867f4bc37841552f P 419cd963fbdb4d7fbb7f8b16833bfb18: Time spent starting tablet: real 0.012s user 0.011s sys 0.001s
I20260806 09:30:02.492535 4261 ts_tablet_manager.cc:1428] T 3afc962059534798867f4bc37841552f P 889f19c67adc423d8abddec2a680acaa: Time spent starting tablet: real 0.015s user 0.012s sys 0.004s
I20260806 09:30:02.493064 4234 heartbeater.cc:503] Master 127.20.113.62:40793 was elected leader, sending a full tablet report...
I20260806 09:30:02.493005 4262 raft_consensus.cc:348] T 3afc962059534798867f4bc37841552f P c86cc3847e78481d84faace4cc1fddba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "889f19c67adc423d8abddec2a680acaa" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 43663 } } peers { permanent_uuid: "c86cc3847e78481d84faace4cc1fddba" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 34241 } } peers { permanent_uuid: "419cd963fbdb4d7fbb7f8b16833bfb18" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 39119 } }
I20260806 09:30:02.493328 4262 raft_consensus.cc:374] T 3afc962059534798867f4bc37841552f P c86cc3847e78481d84faace4cc1fddba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:30:02.493391 4262 raft_consensus.cc:729] T 3afc962059534798867f4bc37841552f P c86cc3847e78481d84faace4cc1fddba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c86cc3847e78481d84faace4cc1fddba, State: Initialized, Role: FOLLOWER
I20260806 09:30:02.493960 4262 consensus_queue.cc:260] T 3afc962059534798867f4bc37841552f P c86cc3847e78481d84faace4cc1fddba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "889f19c67adc423d8abddec2a680acaa" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 43663 } } peers { permanent_uuid: "c86cc3847e78481d84faace4cc1fddba" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 34241 } } peers { permanent_uuid: "419cd963fbdb4d7fbb7f8b16833bfb18" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 39119 } }
I20260806 09:30:02.495693 4262 ts_tablet_manager.cc:1428] T 3afc962059534798867f4bc37841552f P c86cc3847e78481d84faace4cc1fddba: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
W20260806 09:30:02.521725 4042 tablet.cc:2369] T 3afc962059534798867f4bc37841552f P 419cd963fbdb4d7fbb7f8b16833bfb18: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 09:30:02.581863 4277 raft_consensus.cc:482] T 3afc962059534798867f4bc37841552f P c86cc3847e78481d84faace4cc1fddba [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 09:30:02.582262 4277 raft_consensus.cc:504] T 3afc962059534798867f4bc37841552f P c86cc3847e78481d84faace4cc1fddba [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "889f19c67adc423d8abddec2a680acaa" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 43663 } } peers { permanent_uuid: "c86cc3847e78481d84faace4cc1fddba" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 34241 } } peers { permanent_uuid: "419cd963fbdb4d7fbb7f8b16833bfb18" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 39119 } }
I20260806 09:30:02.584115 4277 leader_election.cc:288] T 3afc962059534798867f4bc37841552f P c86cc3847e78481d84faace4cc1fddba [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 889f19c67adc423d8abddec2a680acaa (127.20.113.3:43663), 419cd963fbdb4d7fbb7f8b16833bfb18 (127.20.113.2:39119)
I20260806 09:30:02.593318 4253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:30:02.584594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.1:40243 (local address 127.20.113.3:43663)
0806 09:30:02.584802 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0806 09:30:02.584809 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:30:02.585559 (+ 750us) server_negotiation.cc:407] Connection header received
0806 09:30:02.585778 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:30:02.585783 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:30:02.585862 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:30:02.585971 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:30:02.586906 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:30:02.588041 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:30:02.590944 (+ 2903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:30:02.592144 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:30:02.592331 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:30:02.592403 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 09:30:02.592913 (+ 510us) server_negotiation.cc:299] Negotiation successful
0806 09:30:02.593048 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 09:30:02.593309 4256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:30:02.584711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.1:46339 (local address 127.20.113.2:39119)
0806 09:30:02.584919 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0806 09:30:02.584926 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:30:02.585668 (+ 742us) server_negotiation.cc:407] Connection header received
0806 09:30:02.585777 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:30:02.585783 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:30:02.585862 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:30:02.585972 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:30:02.586914 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:30:02.588045 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:30:02.590894 (+ 2849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:30:02.592141 (+ 1247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:30:02.592321 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:30:02.592395 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 09:30:02.592950 (+ 555us) server_negotiation.cc:299] Negotiation successful
0806 09:30:02.593072 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 09:30:02.593556 4279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:30:02.584653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.2:39119 (local address 127.20.113.1:46339)
0806 09:30:02.585551 (+ 898us) negotiation.cc:107] Waiting for socket to connect
0806 09:30:02.585567 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 09:30:02.585666 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:30:02.586011 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:30:02.586016 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:30:02.586044 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:30:02.586689 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:30:02.586698 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:30:02.588193 (+ 1495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:30:02.588197 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:30:02.590637 (+ 2440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:30:02.590653 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:30:02.592460 (+ 1807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:30:02.592465 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:30:02.592692 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:30:02.592763 (+ 71us) client_negotiation.cc:769] Sending connection context
0806 09:30:02.592863 (+ 100us) client_negotiation.cc:240] Negotiation successful
0806 09:30:02.593112 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":775,"mutex_wait_us":151,"thread_start_us":555,"threads_started":1}
I20260806 09:30:02.593608 4278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:30:02.584547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.3:43663 (local address 127.20.113.1:40243)
0806 09:30:02.585409 (+ 862us) negotiation.cc:107] Waiting for socket to connect
0806 09:30:02.585447 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 09:30:02.585621 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:30:02.586011 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:30:02.586016 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:30:02.586105 (+ 89us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:30:02.586684 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:30:02.586698 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:30:02.588191 (+ 1493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:30:02.588196 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:30:02.590637 (+ 2441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:30:02.590653 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:30:02.592453 (+ 1800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:30:02.592459 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:30:02.592699 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:30:02.592765 (+ 66us) client_negotiation.cc:769] Sending connection context
0806 09:30:02.592863 (+ 98us) client_negotiation.cc:240] Negotiation successful
0806 09:30:02.593008 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":696,"mutex_wait_us":130,"spinlock_wait_cycles":26496,"thread_start_us":447,"threads_started":1}
I20260806 09:30:02.594901 3977 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3afc962059534798867f4bc37841552f" candidate_uuid: "c86cc3847e78481d84faace4cc1fddba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "419cd963fbdb4d7fbb7f8b16833bfb18" is_pre_election: true
I20260806 09:30:02.594937 4166 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3afc962059534798867f4bc37841552f" candidate_uuid: "c86cc3847e78481d84faace4cc1fddba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "889f19c67adc423d8abddec2a680acaa" is_pre_election: true
W20260806 09:30:02.595319 4235 tablet.cc:2369] T 3afc962059534798867f4bc37841552f P 889f19c67adc423d8abddec2a680acaa: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 09:30:02.595578 3977 raft_consensus.cc:2454] T 3afc962059534798867f4bc37841552f P 419cd963fbdb4d7fbb7f8b16833bfb18 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c86cc3847e78481d84faace4cc1fddba in term 0.
I20260806 09:30:02.595582 4166 raft_consensus.cc:2454] T 3afc962059534798867f4bc37841552f P 889f19c67adc423d8abddec2a680acaa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c86cc3847e78481d84faace4cc1fddba in term 0.
I20260806 09:30:02.596387 3708 leader_election.cc:302] T 3afc962059534798867f4bc37841552f P c86cc3847e78481d84faace4cc1fddba [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 419cd963fbdb4d7fbb7f8b16833bfb18, c86cc3847e78481d84faace4cc1fddba; no voters:
I20260806 09:30:02.596820 4277 raft_consensus.cc:2781] T 3afc962059534798867f4bc37841552f P c86cc3847e78481d84faace4cc1fddba [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 09:30:02.596979 4277 raft_consensus.cc:482] T 3afc962059534798867f4bc37841552f P c86cc3847e78481d84faace4cc1fddba [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 09:30:02.597105 4277 raft_consensus.cc:3037] T 3afc962059534798867f4bc37841552f P c86cc3847e78481d84faace4cc1fddba [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:30:02.599581 4277 raft_consensus.cc:504] T 3afc962059534798867f4bc37841552f P c86cc3847e78481d84faace4cc1fddba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "889f19c67adc423d8abddec2a680acaa" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 43663 } } peers { permanent_uuid: "c86cc3847e78481d84faace4cc1fddba" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 34241 } } peers { permanent_uuid: "419cd963fbdb4d7fbb7f8b16833bfb18" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 39119 } }
I20260806 09:30:02.600242 4277 leader_election.cc:288] T 3afc962059534798867f4bc37841552f P c86cc3847e78481d84faace4cc1fddba [CANDIDATE]: Term 1 election: Requested vote from peers 889f19c67adc423d8abddec2a680acaa (127.20.113.3:43663), 419cd963fbdb4d7fbb7f8b16833bfb18 (127.20.113.2:39119)
I20260806 09:30:02.600728 3977 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3afc962059534798867f4bc37841552f" candidate_uuid: "c86cc3847e78481d84faace4cc1fddba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "419cd963fbdb4d7fbb7f8b16833bfb18"
I20260806 09:30:02.600737 4166 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3afc962059534798867f4bc37841552f" candidate_uuid: "c86cc3847e78481d84faace4cc1fddba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "889f19c67adc423d8abddec2a680acaa"
I20260806 09:30:02.600932 3977 raft_consensus.cc:3037] T 3afc962059534798867f4bc37841552f P 419cd963fbdb4d7fbb7f8b16833bfb18 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:30:02.600948 4166 raft_consensus.cc:3037] T 3afc962059534798867f4bc37841552f P 889f19c67adc423d8abddec2a680acaa [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:30:02.603195 4166 raft_consensus.cc:2454] T 3afc962059534798867f4bc37841552f P 889f19c67adc423d8abddec2a680acaa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c86cc3847e78481d84faace4cc1fddba in term 1.
I20260806 09:30:02.603202 3977 raft_consensus.cc:2454] T 3afc962059534798867f4bc37841552f P 419cd963fbdb4d7fbb7f8b16833bfb18 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c86cc3847e78481d84faace4cc1fddba in term 1.
I20260806 09:30:02.603657 3702 leader_election.cc:302] T 3afc962059534798867f4bc37841552f P c86cc3847e78481d84faace4cc1fddba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 889f19c67adc423d8abddec2a680acaa, c86cc3847e78481d84faace4cc1fddba; no voters:
I20260806 09:30:02.604038 4277 raft_consensus.cc:2781] T 3afc962059534798867f4bc37841552f P c86cc3847e78481d84faace4cc1fddba [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:30:02.604589 4277 raft_consensus.cc:686] T 3afc962059534798867f4bc37841552f P c86cc3847e78481d84faace4cc1fddba [term 1 LEADER]: Becoming Leader. State: Replica: c86cc3847e78481d84faace4cc1fddba, State: Running, Role: LEADER
I20260806 09:30:02.605096 4277 consensus_queue.cc:237] T 3afc962059534798867f4bc37841552f P c86cc3847e78481d84faace4cc1fddba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "889f19c67adc423d8abddec2a680acaa" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 43663 } } peers { permanent_uuid: "c86cc3847e78481d84faace4cc1fddba" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 34241 } } peers { permanent_uuid: "419cd963fbdb4d7fbb7f8b16833bfb18" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 39119 } }
I20260806 09:30:02.610165 3586 catalog_manager.cc:5495] T 3afc962059534798867f4bc37841552f P c86cc3847e78481d84faace4cc1fddba reported cstate change: term changed from 0 to 1, leader changed from <none> to c86cc3847e78481d84faace4cc1fddba (127.20.113.1). New cstate: current_term: 1 leader_uuid: "c86cc3847e78481d84faace4cc1fddba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "889f19c67adc423d8abddec2a680acaa" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 43663 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c86cc3847e78481d84faace4cc1fddba" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 34241 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "419cd963fbdb4d7fbb7f8b16833bfb18" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 39119 } health_report { overall_health: UNKNOWN } } }
I20260806 09:30:02.641374 4258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:30:02.635437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47886 (local address 127.20.113.1:34241)
0806 09:30:02.635674 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0806 09:30:02.635682 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 09:30:02.635790 (+ 108us) server_negotiation.cc:407] Connection header received
0806 09:30:02.636077 (+ 287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:30:02.636081 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:30:02.636193 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:30:02.636427 (+ 234us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 09:30:02.636888 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:30:02.637634 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:30:02.639015 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:30:02.639452 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:30:02.639600 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:30:02.640127 (+ 527us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 09:30:02.640677 (+ 550us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 09:30:02.640857 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0806 09:30:02.640980 (+ 123us) server_negotiation.cc:299] Negotiation successful
0806 09:30:02.641074 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 09:30:02.649919 4166 raft_consensus.cc:1261] T 3afc962059534798867f4bc37841552f P 889f19c67adc423d8abddec2a680acaa [term 1 FOLLOWER]: Refusing update from remote peer c86cc3847e78481d84faace4cc1fddba: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 09:30:02.649928 3977 raft_consensus.cc:1261] T 3afc962059534798867f4bc37841552f P 419cd963fbdb4d7fbb7f8b16833bfb18 [term 1 FOLLOWER]: Refusing update from remote peer c86cc3847e78481d84faace4cc1fddba: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 09:30:02.650977 4280 consensus_queue.cc:1035] T 3afc962059534798867f4bc37841552f P c86cc3847e78481d84faace4cc1fddba [LEADER]: Connected to new peer: Peer: permanent_uuid: "889f19c67adc423d8abddec2a680acaa" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 43663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 09:30:02.651337 4277 consensus_queue.cc:1035] T 3afc962059534798867f4bc37841552f P c86cc3847e78481d84faace4cc1fddba [LEADER]: Connected to new peer: Peer: permanent_uuid: "419cd963fbdb4d7fbb7f8b16833bfb18" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 39119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 09:30:02.662194 4284 mvcc.cc:205] Tried to move back new op lower bound from 7315491236443271168 to 7315491236274470912. Current Snapshot: MvccSnapshot[applied={T|T < 7315491236443271168}]
I20260806 09:30:02.667172 4288 mvcc.cc:205] Tried to move back new op lower bound from 7315491236443271168 to 7315491236274470912. Current Snapshot: MvccSnapshot[applied={T|T < 7315491236443271168}]
I20260806 09:30:02.676501 3587 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36542:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 09:30:02.676834 3587 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36542:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 09:30:02.680717 3587 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7c897a4b87744bfb928a75bda9836791: Sending DeleteTablet for 3 replicas of tablet 3afc962059534798867f4bc37841552f
I20260806 09:30:02.681948 4143 tablet_service.cc:1514] Processing DeleteTablet for tablet 3afc962059534798867f4bc37841552f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:30:02 UTC) from {username='kudu', principal='kudu/127.20.113.62@KRBTEST.COM'} at 127.0.0.1:34010
I20260806 09:30:02.682031 3767 tablet_service.cc:1514] Processing DeleteTablet for tablet 3afc962059534798867f4bc37841552f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:30:02 UTC) from {username='kudu', principal='kudu/127.20.113.62@KRBTEST.COM'} at 127.0.0.1:47880
I20260806 09:30:02.683364 3951 tablet_service.cc:1514] Processing DeleteTablet for tablet 3afc962059534798867f4bc37841552f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:30:02 UTC) from {username='kudu', principal='kudu/127.20.113.62@KRBTEST.COM'} at 127.0.0.1:51046
I20260806 09:30:02.685316 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 3678
I20260806 09:30:02.692852 4300 tablet_replica.cc:330] T 3afc962059534798867f4bc37841552f P 889f19c67adc423d8abddec2a680acaa: stopping tablet replica
I20260806 09:30:02.693265 4301 tablet_replica.cc:330] T 3afc962059534798867f4bc37841552f P 419cd963fbdb4d7fbb7f8b16833bfb18: stopping tablet replica
I20260806 09:30:02.694139 4300 raft_consensus.cc:2229] T 3afc962059534798867f4bc37841552f P 889f19c67adc423d8abddec2a680acaa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 09:30:02.694478 4301 raft_consensus.cc:2229] T 3afc962059534798867f4bc37841552f P 419cd963fbdb4d7fbb7f8b16833bfb18 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 09:30:02.694679 4300 raft_consensus.cc:2258] T 3afc962059534798867f4bc37841552f P 889f19c67adc423d8abddec2a680acaa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 09:30:02.695011 4301 raft_consensus.cc:2258] T 3afc962059534798867f4bc37841552f P 419cd963fbdb4d7fbb7f8b16833bfb18 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 09:30:02.695842 4299 tablet_replica.cc:330] T 3afc962059534798867f4bc37841552f P c86cc3847e78481d84faace4cc1fddba: stopping tablet replica
I20260806 09:30:02.696231 4299 raft_consensus.cc:2229] T 3afc962059534798867f4bc37841552f P c86cc3847e78481d84faace4cc1fddba [term 1 LEADER]: Raft consensus shutting down.
I20260806 09:30:02.697023 4299 raft_consensus.cc:2258] T 3afc962059534798867f4bc37841552f P c86cc3847e78481d84faace4cc1fddba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 09:30:02.697628 4300 ts_tablet_manager.cc:1905] T 3afc962059534798867f4bc37841552f P 889f19c67adc423d8abddec2a680acaa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 09:30:02.698061 4301 ts_tablet_manager.cc:1905] T 3afc962059534798867f4bc37841552f P 419cd963fbdb4d7fbb7f8b16833bfb18: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 09:30:02.705173 4299 ts_tablet_manager.cc:1905] T 3afc962059534798867f4bc37841552f P c86cc3847e78481d84faace4cc1fddba: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 09:30:02.705611 4300 ts_tablet_manager.cc:1918] T 3afc962059534798867f4bc37841552f P 889f19c67adc423d8abddec2a680acaa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 09:30:02.705798 4300 log.cc:1196] T 3afc962059534798867f4bc37841552f P 889f19c67adc423d8abddec2a680acaa: Deleting WAL directory at /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786008495763054-20932-0/minicluster-data/ts-2/wal/wals/3afc962059534798867f4bc37841552f
I20260806 09:30:02.706004 4301 ts_tablet_manager.cc:1918] T 3afc962059534798867f4bc37841552f P 419cd963fbdb4d7fbb7f8b16833bfb18: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 09:30:02.706251 4301 log.cc:1196] T 3afc962059534798867f4bc37841552f P 419cd963fbdb4d7fbb7f8b16833bfb18: Deleting WAL directory at /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786008495763054-20932-0/minicluster-data/ts-1/wal/wals/3afc962059534798867f4bc37841552f
I20260806 09:30:02.706384 4300 ts_tablet_manager.cc:1939] T 3afc962059534798867f4bc37841552f P 889f19c67adc423d8abddec2a680acaa: Deleting consensus metadata
I20260806 09:30:02.706903 4301 ts_tablet_manager.cc:1939] T 3afc962059534798867f4bc37841552f P 419cd963fbdb4d7fbb7f8b16833bfb18: Deleting consensus metadata
I20260806 09:30:02.708525 3569 catalog_manager.cc:4841] TS 889f19c67adc423d8abddec2a680acaa (127.20.113.3:43663): tablet 3afc962059534798867f4bc37841552f (table test-table [id=fbf8afa0663c47e3b1a181f327a1d48f]) successfully deleted
I20260806 09:30:02.708722 3571 catalog_manager.cc:4841] TS 419cd963fbdb4d7fbb7f8b16833bfb18 (127.20.113.2:39119): tablet 3afc962059534798867f4bc37841552f (table test-table [id=fbf8afa0663c47e3b1a181f327a1d48f]) successfully deleted
I20260806 09:30:02.712262 4299 ts_tablet_manager.cc:1918] T 3afc962059534798867f4bc37841552f P c86cc3847e78481d84faace4cc1fddba: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 09:30:02.712527 4299 log.cc:1196] T 3afc962059534798867f4bc37841552f P c86cc3847e78481d84faace4cc1fddba: Deleting WAL directory at /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786008495763054-20932-0/minicluster-data/ts-0/wal/wals/3afc962059534798867f4bc37841552f
I20260806 09:30:02.713147 4299 ts_tablet_manager.cc:1939] T 3afc962059534798867f4bc37841552f P c86cc3847e78481d84faace4cc1fddba: Deleting consensus metadata
Aug 06 09:30:02 dist-test-slave-45dx krb5kdc[3527](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008599, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.113.1@KRBTEST.COM
I20260806 09:30:02.714802 3569 catalog_manager.cc:4841] TS c86cc3847e78481d84faace4cc1fddba (127.20.113.1:34241): tablet 3afc962059534798867f4bc37841552f (table test-table [id=fbf8afa0663c47e3b1a181f327a1d48f]) successfully deleted
I20260806 09:30:02.722337 4258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:30:02.687048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47894 (local address 127.20.113.1:34241)
0806 09:30:02.687231 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0806 09:30:02.687237 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:30:02.694232 (+ 6995us) server_negotiation.cc:407] Connection header received
0806 09:30:02.694434 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:30:02.694440 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:30:02.694550 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:30:02.694656 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:30:02.698726 (+ 4070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:30:02.699868 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:30:02.701880 (+ 2012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:30:02.702613 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:30:02.702713 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:30:02.715626 (+ 12913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:30:02.715639 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:30:02.715654 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:30:02.715703 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:30:02.719059 (+ 3356us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:30:02.720060 (+ 1001us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:30:02.720067 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:30:02.720070 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:30:02.720226 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:30:02.720741 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:30:02.720745 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:30:02.720747 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:30:02.721134 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:30:02.721375 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0806 09:30:02.721801 (+ 426us) server_negotiation.cc:299] Negotiation successful
0806 09:30:02.722000 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 09:30:02.723133 3848 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:30:02.829787 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 3678
I20260806 09:30:02.859647 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 3865
Aug 06 09:30:02 dist-test-slave-45dx krb5kdc[3527](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008599, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.113.2@KRBTEST.COM
I20260806 09:30:02.875771 4256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:30:02.860748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51060 (local address 127.20.113.2:39119)
0806 09:30:02.860986 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0806 09:30:02.860990 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 09:30:02.861012 (+ 22us) server_negotiation.cc:407] Connection header received
0806 09:30:02.861202 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:30:02.861207 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:30:02.861292 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:30:02.861382 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:30:02.862707 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:30:02.863449 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:30:02.864889 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:30:02.865301 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:30:02.865408 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:30:02.869709 (+ 4301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:30:02.869726 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:30:02.869744 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:30:02.869790 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:30:02.873028 (+ 3238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:30:02.873738 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:30:02.873747 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:30:02.873749 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:30:02.873874 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:30:02.874242 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:30:02.874245 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:30:02.874247 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:30:02.874680 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:30:02.874944 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0806 09:30:02.875304 (+ 360us) server_negotiation.cc:299] Negotiation successful
0806 09:30:02.875478 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120}
I20260806 09:30:02.876555 4032 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:30:02.966499 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 3865
I20260806 09:30:02.996011 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 4047
Aug 06 09:30:03 dist-test-slave-45dx krb5kdc[3527](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008599, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.113.3@KRBTEST.COM
I20260806 09:30:03.011144 4253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:30:02.996817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34012 (local address 127.20.113.3:43663)
0806 09:30:02.997000 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0806 09:30:02.997005 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:30:02.997024 (+ 19us) server_negotiation.cc:407] Connection header received
0806 09:30:02.997160 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:30:02.997164 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:30:02.997230 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:30:02.997313 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:30:02.998647 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:30:02.999362 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:30:03.000656 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:30:03.001032 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:30:03.001125 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:30:03.005637 (+ 4512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:30:03.005656 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:30:03.005675 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:30:03.005711 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:30:03.008716 (+ 3005us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:30:03.009355 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:30:03.009362 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:30:03.009364 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:30:03.009506 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:30:03.009840 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:30:03.009843 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:30:03.009845 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:30:03.010208 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:30:03.010430 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0806 09:30:03.010690 (+ 260us) server_negotiation.cc:299] Negotiation successful
0806 09:30:03.010858 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 09:30:03.011792 4227 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:30:03.101603 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 4047
I20260806 09:30:03.129247 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 3546
I20260806 09:30:03.130570 3635 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:30:03.243140 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 3546
2026-08-06T09:30:03Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (4799 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 09:30:03 dist-test-slave-45dx krb5kdc[4311](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 09:30:03 dist-test-slave-45dx krb5kdc[4311](info): set up 2 sockets
Aug 06 09:30:03 dist-test-slave-45dx krb5kdc[4311](info): commencing operation
krb5kdc: starting...
W20260806 09:30:04.582651 20932 mini_kdc.cc:121] Time spent starting KDC: real 1.288s user 0.000s sys 0.053s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/krb5kdc/test-user.keytab.
Aug 06 09:30:04 dist-test-slave-45dx krb5kdc[4311](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008604, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T09:30:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T09:30:04Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.20.113.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.20.113.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.20.113.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/krb5kdc/rangeradmin_127.20.113.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.20.113.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/krb5kdc/rangeradmin_127.20.113.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.20.113.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.20.113.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.20.113.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/krb5kdc/rangerlookup_127.20.113.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.20.113.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/krb5kdc/rangerlookup_127.20.113.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.20.113.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.20.113.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.113.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/krb5kdc/HTTP_127.20.113.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.20.113.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/krb5kdc/HTTP_127.20.113.20@KRBTEST.COM.keytab.
I20260806 09:30:05.063993 20932 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskKR4MEC/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/minicluster-data/postgres -l logfile start
2026-08-06 09:30:07.712 UTC [4349] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-06 09:30:07.712 UTC [4349] LOG: listening on IPv4 address "127.20.113.20", port 44841
2026-08-06 09:30:07.716 UTC [4349] LOG: listening on Unix socket "/tmp/.s.PGSQL.44841"
2026-08-06 09:30:07.743 UTC [4351] LOG: database system was shut down at 2026-08-06 09:30:06 UTC
2026-08-06 09:30:07.747 UTC [4349] LOG: database system is ready to accept connections
I20260806 09:30:08.920722 20932 mini_postgres.cc:96] Postgres bound to 44841
2026-08-06 09:30:08.950 UTC [4362] FATAL: database "slave" does not exist
127.20.113.20:44841 - accepting connections
I20260806 09:30:08.951593 20932 mini_ranger.cc:161] Starting Ranger...
I20260806 09:30:08.986281 20932 mini_ranger.cc:84] Created miniranger Postgres user
I20260806 09:30:09.469718 20932 mini_ranger.cc:87] Created ranger Postgres database
I20260806 09:30:09.469882 20932 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/minicluster-data/ranger-admin
2026-08-06 09:30:09,865 [I] DB FLAVOR :POSTGRES
2026-08-06 09:30:09,866 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 09:30:09,866 [I] Checking connection..
2026-08-06 09:30:09,866 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKR4MEC/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKR4MEC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.113.20:44841/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-06 09:30:10,160 [I] Checking connection passed.
2026-08-06 09:30:10,160 [I] --------- Verifying version history table ---------
2026-08-06 09:30:10,161 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKR4MEC/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKR4MEC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.113.20:44841/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 09:30:10,424 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 09:30:10,424 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKR4MEC/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKR4MEC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.113.20:44841/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 09:30:10,699 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 09:30:10,699 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-06 09:30:10,700 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKR4MEC/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKR4MEC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.113.20:44841/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskKR4MEC/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-06 09:30:10.963 UTC [4455] WARNING: there is no transaction in progress
2026-08-06 09:30:10,970 [I] create_dbversion_catalog.sql file imported successfully
2026-08-06 09:30:10,971 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKR4MEC/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKR4MEC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.113.20:44841/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 09:30:11,234 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-06 09:30:11,235 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 09:30:11,235 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKR4MEC/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKR4MEC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.113.20:44841/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-06 09:30:11,495 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKR4MEC/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKR4MEC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.113.20:44841/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-06 09:30:11,751 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKR4MEC/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKR4MEC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.113.20:44841/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-45dx.c.gcp-upstream.internal','N') ;"
2026-08-06 09:30:12,007 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKR4MEC/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKR4MEC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.113.20:44841/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 09:30:12,252 [I] Table x_portal_user does not exist in database ranger
2026-08-06 09:30:12,253 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKR4MEC/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKR4MEC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.113.20:44841/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 09:30:12,524 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-06 09:30:12,524 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKR4MEC/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKR4MEC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.113.20:44841/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 09:30:12,787 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKR4MEC/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKR4MEC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.113.20:44841/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 09:30:13,047 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-06 09:30:13,047 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKR4MEC/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKR4MEC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.113.20:44841/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskKR4MEC/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-06 09:30:14.040 UTC [4607] WARNING: there is no transaction in progress
2026-08-06 09:30:14.056 UTC [4607] WARNING: there is no transaction in progress
2026-08-06 09:30:14.071 UTC [4607] WARNING: there is no transaction in progress
2026-08-06 09:30:14.086 UTC [4607] WARNING: there is no transaction in progress
2026-08-06 09:30:14.102 UTC [4607] WARNING: there is no transaction in progress
2026-08-06 09:30:14.123 UTC [4607] WARNING: there is no transaction in progress
2026-08-06 09:30:14.185 UTC [4607] WARNING: there is no transaction in progress
2026-08-06 09:30:14.192 UTC [4607] WARNING: there is no transaction in progress
2026-08-06 09:30:14.205 UTC [4607] WARNING: there is no transaction in progress
2026-08-06 09:30:14.215 UTC [4607] WARNING: there is no transaction in progress
2026-08-06 09:30:14.225 UTC [4607] WARNING: there is no transaction in progress
2026-08-06 09:30:14.239 UTC [4607] WARNING: there is no transaction in progress
2026-08-06 09:30:14.248 UTC [4607] WARNING: there is no transaction in progress
2026-08-06 09:30:14.254 UTC [4607] WARNING: there is no transaction in progress
2026-08-06 09:30:14.262 UTC [4607] WARNING: there is no transaction in progress
2026-08-06 09:30:14.269 UTC [4607] WARNING: there is no transaction in progress
2026-08-06 09:30:14,902 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-06 09:30:14,902 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKR4MEC/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKR4MEC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.113.20:44841/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-06 09:30:15,178 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-06 09:30:15,178 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKR4MEC/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKR4MEC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.113.20:44841/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 09:30:15,463 [I] Table x_portal_user already exists in database 'ranger'
2026-08-06 09:30:15,464 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKR4MEC/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKR4MEC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.113.20:44841/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 09:30:15,721 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-06 09:30:15,721 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKR4MEC/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKR4MEC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.113.20:44841/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 09:30:15,959 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKR4MEC/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKR4MEC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.113.20:44841/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 09:30:16,221 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKR4MEC/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKR4MEC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.113.20:44841/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-45dx.c.gcp-upstream.internal';"
2026-08-06 09:30:16,467 [I] CORE_DB_SCHEMA import status has been updated
2026-08-06 09:30:16,468 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKR4MEC/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKR4MEC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.113.20:44841/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-06 09:30:16,736 [I] DB_PATCHES have already been applied
I20260806 09:30:16.743686 20932 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/minicluster-data/ranger-admin:/tmp/dist-test-taskKR4MEC/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskKR4MEC/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskKR4MEC/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKR4MEC/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260806 09:30:16.743811 20932 mini_ranger.cc:193] Using host: 127.20.113.20
I20260806 09:30:16.769913 20932 mini_ranger.cc:227] Ranger admin URL: http://127.20.113.20:42517
Aug 06, 2026 9:30:17 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 9:30:17 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 9:30:17 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/minicluster-data/ranger-admin/ews/webapp
Aug 06, 2026 9:30:17 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 06, 2026 9:30:17 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 06, 2026 9:30:17 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/minicluster-data/ranger-admin/ews/webapp].
Aug 06, 2026 9:30:17 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260806 09:30:18.137065 20932 mini_ranger.cc:160] Time spent starting Ranger: real 9.186s user 0.007s sys 0.072s
Aug 06, 2026 9:30:38 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 06, 2026 9:30:38 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 06, 2026 9:30:38 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 06, 2026 9:30:38 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 06, 2026 9:30:38 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 06, 2026 9:30:38 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 06, 2026 9:30:38 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 06, 2026 9:30:38 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 06, 2026 9:30:38 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 06, 2026 9:30:38 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 06, 2026 9:30:38 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 06, 2026 9:30:38 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 06, 2026 9:30:38 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 06, 2026 9:30:38 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 06, 2026 9:30:38 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 06, 2026 9:30:38 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 06, 2026 9:30:38 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 06, 2026 9:30:39 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260806 09:30:39.792572 20932 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.20.113.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.20.113.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.20.113.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/krb5kdc/rangerkms_127.20.113.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.20.113.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/krb5kdc/rangerkms_127.20.113.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.20.113.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.20.113.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.113.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/krb5kdc/HTTP_127.20.113.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.20.113.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/krb5kdc/HTTP_127.20.113.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 06 09:30:40 dist-test-slave-45dx krb5kdc[4311](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008640, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260806 09:30:40.069412 20932 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260806 09:30:40.102156 20932 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260806 09:30:40.924252 20932 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260806 09:30:40.924487 20932 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/minicluster-data/ranger-kms
I20260806 09:30:40.924642 20932 mini_ranger_kms.cc:227] Using postgres at 127.20.113.20:44841
2026-08-06 09:30:41,457 [I] DB FLAVOR :POSTGRES
2026-08-06 09:30:41,457 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 09:30:41,457 [I] Checking connection
2026-08-06 09:30:41,458 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKR4MEC/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.113.20:44841/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-06 09:30:41,723 [I] connection success
2026-08-06 09:30:41,723 [I] --------- Verifying Ranger DB tables ---------
2026-08-06 09:30:41,723 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-06 09:30:41,723 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKR4MEC/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.113.20:44841/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-06 09:30:41,992 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-06 09:30:41,992 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 09:30:41,992 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-06 09:30:41,993 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKR4MEC/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.113.20:44841/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-06 09:30:42,254 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260806 09:30:42.752343 20932 mini_ranger_kms.cc:310] Created kms service
I20260806 09:30:42.920878 20932 mini_ranger_kms.cc:326] Created kudu user
I20260806 09:30:42.968573 20932 mini_ranger_kms.cc:343] Created rangerkms user
Aug 06 09:30:42 dist-test-slave-45dx krb5kdc[4311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008640, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.20.113.20@KRBTEST.COM
I20260806 09:30:43.361878 20932 mini_ranger_kms.cc:384] Added ranger policy
I20260806 09:30:43.363185 20932 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/minicluster-data/ranger-kms:/tmp/dist-test-taskKR4MEC/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKR4MEC/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskKR4MEC/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260806 09:30:43.363276 20932 mini_ranger_kms.cc:240] Using host: 127.20.113.20
I20260806 09:30:43.396662 20932 mini_ranger_kms.cc:276] Ranger KMS PID: 4867
I20260806 09:30:43.396862 20932 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.20.113.20:40193
Aug 06, 2026 9:30:44 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 9:30:44 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 9:30:44 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 06, 2026 9:30:44 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 06, 2026 9:30:44 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/minicluster-data/ranger-kms/ews/webapp].
Aug 06, 2026 9:30:44 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 06 09:30:45 dist-test-slave-45dx krb5kdc[4311](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786008645, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.20.113.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:30:46.156167 20932 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.087s user 0.143s sys 0.454s
I20260806 09:30:46.156559 20932 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260806 09:30:46.156801 20932 mini_ranger_kms.cc:401] 127.20.113.20:40193/kms/v1/keys
Aug 06 09:30:46 dist-test-slave-45dx krb5kdc[4311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008645, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.20.113.20@KRBTEST.COM for HTTP/127.20.113.20@KRBTEST.COM
Aug 06 09:30:46 dist-test-slave-45dx krb5kdc[4311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008645, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.20.113.20@KRBTEST.COM for HTTP/127.20.113.20@KRBTEST.COM
Aug 06 09:30:49 dist-test-slave-45dx krb5kdc[4311](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008649, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.20.113.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.113.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.113.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab.
Entry for principal kudu/127.20.113.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab.
I20260806 09:30:49.214231 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.113.62:46741
--webserver_interface=127.20.113.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab
--principal=kudu/127.20.113.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:38379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.113.62:46741
--ranger_config_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.113.20:40193/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:30:49.626753 4938 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:30:49.627086 4938 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:30:49.627259 4938 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:30:49.634824 4938 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 09:30:49.634938 4938 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:30:49.635046 4938 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:30:49.635144 4938 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 09:30:49.635231 4938 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 09:30:49.645857 4938 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:38379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/minicluster-data/master-0/wal
--ranger_kms_url=127.20.113.20:40193/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.20.113.62:46741
--ranger_config_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab
--principal=kudu/127.20.113.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.113.62:46741
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.113.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:30:49.647158 4938 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:30:49.648520 4938 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:30:49.661489 4944 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:30:49.661588 4943 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:30:49.661561 4948 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:30:49.662400 4938 server_base.cc:1034] running on GCE node
I20260806 09:30:49.663233 4938 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:30:49.664696 4938 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:30:49.665930 4938 hybrid_clock.cc:648] HybridClock initialized: now 1786008649665899 us; error 44 us; skew 500 ppm
I20260806 09:30:49.666328 4938 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:30:49 dist-test-slave-45dx krb5kdc[4311](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008649, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:30:49.673444 4938 init.cc:377] Logged in from keytab as kudu/127.20.113.62@KRBTEST.COM (short username kudu)
I20260806 09:30:49.675007 4938 webserver.cc:460] Webserver started at http://127.20.113.62:41355/ using document root <none> and password file <none>
I20260806 09:30:49.675741 4938 fs_manager.cc:359] Metadata directory not provided
I20260806 09:30:49.675897 4938 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:30:49.676333 4938 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 09:30:49 dist-test-slave-45dx krb5kdc[4311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008649, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.62@KRBTEST.COM for HTTP/127.20.113.20@KRBTEST.COM
I20260806 09:30:49.726910 4938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/minicluster-data/master-0/data/instance:
uuid: "4334efcb0e8a4e9cb41de47b21ce6f4e"
format_stamp: "Formatted at 2026-08-06 09:30:49 on dist-test-slave-45dx"
server_key: "ade1a8b1fb9d283f45786e130c4a53c8"
server_key_iv: "5b4c29cf49a8f1842530b571616af989"
server_key_version: "kuduclusterkey@0"
I20260806 09:30:49.727903 4938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/minicluster-data/master-0/wal/instance:
uuid: "4334efcb0e8a4e9cb41de47b21ce6f4e"
format_stamp: "Formatted at 2026-08-06 09:30:49 on dist-test-slave-45dx"
server_key: "ade1a8b1fb9d283f45786e130c4a53c8"
server_key_iv: "5b4c29cf49a8f1842530b571616af989"
server_key_version: "kuduclusterkey@0"
I20260806 09:30:49.733857 4938 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260806 09:30:49.755162 4958 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:30:49.757046 4938 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260806 09:30:49.757220 4938 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/minicluster-data/master-0/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/minicluster-data/master-0/wal
uuid: "4334efcb0e8a4e9cb41de47b21ce6f4e"
format_stamp: "Formatted at 2026-08-06 09:30:49 on dist-test-slave-45dx"
server_key: "ade1a8b1fb9d283f45786e130c4a53c8"
server_key_iv: "5b4c29cf49a8f1842530b571616af989"
server_key_version: "kuduclusterkey@0"
I20260806 09:30:49.757426 4938 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:30:49.772845 4938 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:30:49.776958 4938 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:30:49.777275 4938 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:30:49.797252 4938 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.62:46741
I20260806 09:30:49.797361 5030 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.62:46741 every 8 connection(s)
I20260806 09:30:49.798640 4938 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
I20260806 09:30:49.802546 5031 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:30:49.808768 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 4938
I20260806 09:30:49.809100 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/minicluster-data/master-0/wal/instance
I20260806 09:30:49.810125 20932 external_mini_cluster.cc:1428] Setting key 87cb829bd1b702156f524439266079e2
I20260806 09:30:49.814185 5031 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4334efcb0e8a4e9cb41de47b21ce6f4e: Bootstrap starting.
I20260806 09:30:49.818063 5031 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4334efcb0e8a4e9cb41de47b21ce6f4e: Neither blocks nor log segments found. Creating new log.
I20260806 09:30:49.819242 5031 log.cc:824] T 00000000000000000000000000000000 P 4334efcb0e8a4e9cb41de47b21ce6f4e: Log is configured to *not* fsync() on all Append() calls
I20260806 09:30:49.822469 5031 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4334efcb0e8a4e9cb41de47b21ce6f4e: No bootstrap required, opened a new log
Aug 06 09:30:49 dist-test-slave-45dx krb5kdc[4311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008649, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.113.62@KRBTEST.COM
I20260806 09:30:49.830590 5031 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4334efcb0e8a4e9cb41de47b21ce6f4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4334efcb0e8a4e9cb41de47b21ce6f4e" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 46741 } }
I20260806 09:30:49.830991 5031 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4334efcb0e8a4e9cb41de47b21ce6f4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:30:49.831120 5031 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4334efcb0e8a4e9cb41de47b21ce6f4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4334efcb0e8a4e9cb41de47b21ce6f4e, State: Initialized, Role: FOLLOWER
I20260806 09:30:49.831779 5031 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4334efcb0e8a4e9cb41de47b21ce6f4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4334efcb0e8a4e9cb41de47b21ce6f4e" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 46741 } }
I20260806 09:30:49.832098 5031 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4334efcb0e8a4e9cb41de47b21ce6f4e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 09:30:49.832255 5031 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4334efcb0e8a4e9cb41de47b21ce6f4e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 09:30:49.832515 5031 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4334efcb0e8a4e9cb41de47b21ce6f4e [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:30:49.833434 5034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:30:49.812283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56656 (local address 127.20.113.62:46741)
0806 09:30:49.813166 (+ 883us) server_negotiation.cc:206] Beginning negotiation
0806 09:30:49.813177 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 09:30:49.813218 (+ 41us) server_negotiation.cc:407] Connection header received
0806 09:30:49.814981 (+ 1763us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:30:49.815003 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:30:49.815397 (+ 394us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:30:49.815787 (+ 390us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:30:49.817530 (+ 1743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:30:49.818858 (+ 1328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:30:49.820495 (+ 1637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:30:49.821150 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:30:49.821278 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:30:49.825968 (+ 4690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:30:49.825975 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:30:49.825989 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:30:49.826034 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:30:49.829163 (+ 3129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:30:49.829838 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:30:49.829848 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:30:49.829853 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:30:49.830013 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:30:49.830414 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:30:49.830423 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:30:49.830425 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:30:49.831117 (+ 692us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:30:49.831459 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0806 09:30:49.831886 (+ 427us) server_negotiation.cc:299] Negotiation successful
0806 09:30:49.832343 (+ 457us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":622,"thread_start_us":396,"threads_started":1}
I20260806 09:30:49.835021 5031 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4334efcb0e8a4e9cb41de47b21ce6f4e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4334efcb0e8a4e9cb41de47b21ce6f4e" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 46741 } }
I20260806 09:30:49.835533 5031 leader_election.cc:302] T 00000000000000000000000000000000 P 4334efcb0e8a4e9cb41de47b21ce6f4e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4334efcb0e8a4e9cb41de47b21ce6f4e; no voters:
I20260806 09:30:49.836359 5031 leader_election.cc:288] T 00000000000000000000000000000000 P 4334efcb0e8a4e9cb41de47b21ce6f4e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 09:30:49.836463 5039 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4334efcb0e8a4e9cb41de47b21ce6f4e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:30:49.837355 5039 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4334efcb0e8a4e9cb41de47b21ce6f4e [term 1 LEADER]: Becoming Leader. State: Replica: 4334efcb0e8a4e9cb41de47b21ce6f4e, State: Running, Role: LEADER
I20260806 09:30:49.838110 5039 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4334efcb0e8a4e9cb41de47b21ce6f4e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4334efcb0e8a4e9cb41de47b21ce6f4e" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 46741 } }
I20260806 09:30:49.838886 5031 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4334efcb0e8a4e9cb41de47b21ce6f4e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 09:30:49.841176 5041 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4334efcb0e8a4e9cb41de47b21ce6f4e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4334efcb0e8a4e9cb41de47b21ce6f4e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4334efcb0e8a4e9cb41de47b21ce6f4e" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 46741 } } }
I20260806 09:30:49.841759 5041 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4334efcb0e8a4e9cb41de47b21ce6f4e [sys.catalog]: This master's current role is: LEADER
I20260806 09:30:49.841871 5043 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4334efcb0e8a4e9cb41de47b21ce6f4e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4334efcb0e8a4e9cb41de47b21ce6f4e. Latest consensus state: current_term: 1 leader_uuid: "4334efcb0e8a4e9cb41de47b21ce6f4e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4334efcb0e8a4e9cb41de47b21ce6f4e" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 46741 } } }
I20260806 09:30:49.842224 5043 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4334efcb0e8a4e9cb41de47b21ce6f4e [sys.catalog]: This master's current role is: LEADER
I20260806 09:30:49.847493 5047 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 09:30:49.852849 5047 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 09:30:49.853722 5031 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 09:30:49.874691 5047 catalog_manager.cc:1347] Generated new cluster ID: 7e9daeb80dab47bf81af5c3baea3a8d7
I20260806 09:30:49.874873 5047 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 09:30:49.890695 5047 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 09:30:49.892102 5047 catalog_manager.cc:1504] Loading token signing keys...
I20260806 09:30:49.900822 5047 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4334efcb0e8a4e9cb41de47b21ce6f4e: Generated new TSK 0
I20260806 09:30:49.901787 5047 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.113.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.113.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.113.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.1.keytab.
Entry for principal kudu/127.20.113.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.1.keytab.
I20260806 09:30:50.773051 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.113.20:40193/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.113.1:0
--local_ip_for_outbound_sockets=127.20.113.1
--webserver_interface=127.20.113.1
--webserver_port=0
--tserver_master_addrs=127.20.113.62:46741
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.1.keytab
--principal=kudu/127.20.113.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:38379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 09:30:51 dist-test-slave-45dx krb5kdc[4311](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786008651, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260806 09:30:51.256111 5093 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:30:51.256523 5093 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:30:51.256683 5093 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:30:51.268162 5093 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:30:51.268277 5093 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:30:51.268496 5093 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.113.1
I20260806 09:30:51.278981 5093 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:38379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.20.113.20:40193/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.1.keytab
--principal=kudu/127.20.113.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.113.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.113.1
--webserver_port=0
--tserver_master_addrs=127.20.113.62:46741
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.113.1
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:30:51.280526 5093 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:30:51.281996 5093 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:30:51.296069 5108 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:30:51.296114 5106 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:30:51.305326 5111 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:30:51.305732 5093 server_base.cc:1034] running on GCE node
I20260806 09:30:51.306582 5093 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:30:51.307869 5093 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:30:51.309181 5093 hybrid_clock.cc:648] HybridClock initialized: now 1786008651309068 us; error 100 us; skew 500 ppm
I20260806 09:30:51.309618 5093 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:30:51 dist-test-slave-45dx krb5kdc[4311](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008651, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:30:51.314579 5093 init.cc:377] Logged in from keytab as kudu/127.20.113.1@KRBTEST.COM (short username kudu)
I20260806 09:30:51.319823 5093 webserver.cc:460] Webserver started at http://127.20.113.1:41895/ using document root <none> and password file <none>
I20260806 09:30:51.320863 5093 fs_manager.cc:359] Metadata directory not provided
I20260806 09:30:51.321018 5093 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:30:51.321458 5093 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 09:30:51 dist-test-slave-45dx krb5kdc[4311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008651, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.1@KRBTEST.COM for HTTP/127.20.113.20@KRBTEST.COM
I20260806 09:30:51.346272 5093 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/minicluster-data/ts-0/data/instance:
uuid: "4c258f17206b468a9e0572cb7d716049"
format_stamp: "Formatted at 2026-08-06 09:30:51 on dist-test-slave-45dx"
server_key: "39f11bc13eb7a8a64ad1dc0a9cbe2713"
server_key_iv: "8e9819bf2e154ed9426f93abdfe09603"
server_key_version: "kuduclusterkey@0"
I20260806 09:30:51.347771 5093 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/minicluster-data/ts-0/wal/instance:
uuid: "4c258f17206b468a9e0572cb7d716049"
format_stamp: "Formatted at 2026-08-06 09:30:51 on dist-test-slave-45dx"
server_key: "39f11bc13eb7a8a64ad1dc0a9cbe2713"
server_key_iv: "8e9819bf2e154ed9426f93abdfe09603"
server_key_version: "kuduclusterkey@0"
I20260806 09:30:51.362601 5093 fs_manager.cc:691] Time spent creating directory manager: real 0.014s user 0.009s sys 0.003s
I20260806 09:30:51.417485 5128 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:30:51.420158 5093 fs_manager.cc:725] Time spent opening block manager: real 0.026s user 0.010s sys 0.009s
I20260806 09:30:51.420553 5093 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/minicluster-data/ts-0/wal
uuid: "4c258f17206b468a9e0572cb7d716049"
format_stamp: "Formatted at 2026-08-06 09:30:51 on dist-test-slave-45dx"
server_key: "39f11bc13eb7a8a64ad1dc0a9cbe2713"
server_key_iv: "8e9819bf2e154ed9426f93abdfe09603"
server_key_version: "kuduclusterkey@0"
I20260806 09:30:51.420969 5093 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:30:51.468997 5093 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:30:51.475121 5093 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:30:51.475519 5093 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:30:51.479084 5093 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:30:51.481637 5093 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:30:51.481856 5093 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260806 09:30:51.482121 5093 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:30:51.482266 5093 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 09:30:51 dist-test-slave-45dx krb5kdc[4311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008651, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.62@KRBTEST.COM for HTTP/127.20.113.20@KRBTEST.COM
I20260806 09:30:51.703950 5093 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.1:45345
I20260806 09:30:51.704035 5300 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.1:45345 every 8 connection(s)
I20260806 09:30:51.705636 5093 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/minicluster-data/ts-0/data/info.pb
I20260806 09:30:51.711457 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 5093
I20260806 09:30:51.711711 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/minicluster-data/ts-0/wal/instance
I20260806 09:30:51.712774 20932 external_mini_cluster.cc:1428] Setting key 13db31eb149d828c60fbf620b6940d39
Aug 06 09:30:51 dist-test-slave-45dx krb5kdc[4311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008651, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.1@KRBTEST.COM for kudu/127.20.113.62@KRBTEST.COM
I20260806 09:30:51.752631 5304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:30:51.708842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.1:37897 (local address 127.20.113.62:46741)
0806 09:30:51.716920 (+ 8078us) server_negotiation.cc:206] Beginning negotiation
0806 09:30:51.716930 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 09:30:51.729219 (+ 12289us) server_negotiation.cc:407] Connection header received
0806 09:30:51.731675 (+ 2456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:30:51.731682 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:30:51.731820 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:30:51.731912 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:30:51.733984 (+ 2072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:30:51.735269 (+ 1285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:30:51.737917 (+ 2648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:30:51.738721 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:30:51.738848 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:30:51.744000 (+ 5152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:30:51.744009 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:30:51.744016 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:30:51.744074 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:30:51.748332 (+ 4258us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:30:51.749390 (+ 1058us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:30:51.749398 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:30:51.749403 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:30:51.749567 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:30:51.750135 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:30:51.750143 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:30:51.750148 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:30:51.750564 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:30:51.750703 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0806 09:30:51.751769 (+ 1066us) server_negotiation.cc:299] Negotiation successful
0806 09:30:51.752066 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7897,"thread_start_us":7741,"threads_started":1}
I20260806 09:30:51.755535 5305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:30:51.709439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:46741 (local address 127.20.113.1:37897)
0806 09:30:51.729003 (+ 19564us) negotiation.cc:107] Waiting for socket to connect
0806 09:30:51.729062 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0806 09:30:51.731289 (+ 2227us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:30:51.732586 (+ 1297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:30:51.732602 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:30:51.733166 (+ 564us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:30:51.733767 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:30:51.733814 (+ 47us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:30:51.735493 (+ 1679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:30:51.735501 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:30:51.737748 (+ 2247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:30:51.737765 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:30:51.739244 (+ 1479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:30:51.739253 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:30:51.739603 (+ 350us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:30:51.740536 (+ 933us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:30:51.740569 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:30:51.743741 (+ 3172us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:30:51.748516 (+ 4775us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:30:51.748527 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:30:51.748547 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:30:51.749203 (+ 656us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:30:51.749712 (+ 509us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:30:51.749718 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:30:51.749724 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:30:51.749999 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:30:51.750811 (+ 812us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:30:51.750822 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:30:51.751304 (+ 482us) client_negotiation.cc:769] Sending connection context
0806 09:30:51.753359 (+ 2055us) client_negotiation.cc:240] Negotiation successful
0806 09:30:51.753948 (+ 589us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19222,"thread_start_us":18993,"threads_started":1}
WARNING: no policy specified for kudu/127.20.113.2@KRBTEST.COM; defaulting to no policy
I20260806 09:30:51.757633 5301 heartbeater.cc:347] Connected to a master server at 127.20.113.62:46741
I20260806 09:30:51.757967 5301 heartbeater.cc:464] Registering TS with master...
I20260806 09:30:51.758852 5301 heartbeater.cc:511] Master 127.20.113.62:46741 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.113.2@KRBTEST.COM" created.
I20260806 09:30:51.761566 4981 ts_manager.cc:194] Registered new tserver with Master: 4c258f17206b468a9e0572cb7d716049 (127.20.113.1:45345)
I20260806 09:30:51.763700 4981 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.113.1@KRBTEST.COM'} at 127.20.113.1:37897
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.113.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.2.keytab.
Entry for principal kudu/127.20.113.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.2.keytab.
I20260806 09:30:51.798421 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.113.20:40193/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.113.2:0
--local_ip_for_outbound_sockets=127.20.113.2
--webserver_interface=127.20.113.2
--webserver_port=0
--tserver_master_addrs=127.20.113.62:46741
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.2.keytab
--principal=kudu/127.20.113.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:38379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 09:30:51 dist-test-slave-45dx krb5kdc[4311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008651, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.62@KRBTEST.COM for HTTP/127.20.113.20@KRBTEST.COM
W20260806 09:30:52.289999 5308 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:30:52.290458 5308 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:30:52.290791 5308 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:30:52.303354 5308 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:30:52.303469 5308 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:30:52.303674 5308 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.113.2
I20260806 09:30:52.314054 5308 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:38379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.20.113.20:40193/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.2.keytab
--principal=kudu/127.20.113.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.113.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.113.2
--webserver_port=0
--tserver_master_addrs=127.20.113.62:46741
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.113.2
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:30:52.315430 5308 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:30:52.316849 5308 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:30:52.331067 5327 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:30:52.331108 5323 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:30:52.331154 5324 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:30:52.333900 5308 server_base.cc:1034] running on GCE node
I20260806 09:30:52.334600 5308 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:30:52.335632 5308 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:30:52.336876 5308 hybrid_clock.cc:648] HybridClock initialized: now 1786008652336824 us; error 58 us; skew 500 ppm
I20260806 09:30:52.337186 5308 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:30:52 dist-test-slave-45dx krb5kdc[4311](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008652, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:30:52.340787 5308 init.cc:377] Logged in from keytab as kudu/127.20.113.2@KRBTEST.COM (short username kudu)
I20260806 09:30:52.342439 5308 webserver.cc:460] Webserver started at http://127.20.113.2:33057/ using document root <none> and password file <none>
I20260806 09:30:52.343215 5308 fs_manager.cc:359] Metadata directory not provided
I20260806 09:30:52.343371 5308 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:30:52.343703 5308 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 09:30:52 dist-test-slave-45dx krb5kdc[4311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008652, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.2@KRBTEST.COM for HTTP/127.20.113.20@KRBTEST.COM
I20260806 09:30:52.360385 5308 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/minicluster-data/ts-1/data/instance:
uuid: "13d78464f14b4e67a40b713e799f1ba1"
format_stamp: "Formatted at 2026-08-06 09:30:52 on dist-test-slave-45dx"
server_key: "ad7819668a26611190a9ea97a8739ffe"
server_key_iv: "2b3319003c9d6bb5d97a21a2eac8418b"
server_key_version: "kuduclusterkey@0"
I20260806 09:30:52.361611 5308 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/minicluster-data/ts-1/wal/instance:
uuid: "13d78464f14b4e67a40b713e799f1ba1"
format_stamp: "Formatted at 2026-08-06 09:30:52 on dist-test-slave-45dx"
server_key: "ad7819668a26611190a9ea97a8739ffe"
server_key_iv: "2b3319003c9d6bb5d97a21a2eac8418b"
server_key_version: "kuduclusterkey@0"
I20260806 09:30:52.369294 5308 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.001s
I20260806 09:30:52.385414 5338 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:30:52.387269 5308 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 09:30:52.387506 5308 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/minicluster-data/ts-1/wal
uuid: "13d78464f14b4e67a40b713e799f1ba1"
format_stamp: "Formatted at 2026-08-06 09:30:52 on dist-test-slave-45dx"
server_key: "ad7819668a26611190a9ea97a8739ffe"
server_key_iv: "2b3319003c9d6bb5d97a21a2eac8418b"
server_key_version: "kuduclusterkey@0"
I20260806 09:30:52.387846 5308 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:30:52.410360 5308 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:30:52.415555 5308 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:30:52.415908 5308 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:30:52.417299 5308 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:30:52.418919 5308 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:30:52.419030 5308 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:30:52.419195 5308 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:30:52.419273 5308 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:30:52.467849 5308 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.2:35221
I20260806 09:30:52.467981 5493 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.2:35221 every 8 connection(s)
I20260806 09:30:52.469276 5308 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/minicluster-data/ts-1/data/info.pb
I20260806 09:30:52.473544 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 5308
I20260806 09:30:52.473738 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/minicluster-data/ts-1/wal/instance
I20260806 09:30:52.474431 20932 external_mini_cluster.cc:1428] Setting key 8752334ca00c4b3bba83c0bd8259b5d4
Aug 06 09:30:52 dist-test-slave-45dx krb5kdc[4311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008652, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.2@KRBTEST.COM for kudu/127.20.113.62@KRBTEST.COM
I20260806 09:30:52.492703 5497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:30:52.472202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.2:59403 (local address 127.20.113.62:46741)
0806 09:30:52.472722 (+ 520us) server_negotiation.cc:206] Beginning negotiation
0806 09:30:52.472727 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:30:52.474189 (+ 1462us) server_negotiation.cc:407] Connection header received
0806 09:30:52.476775 (+ 2586us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:30:52.476780 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:30:52.476886 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:30:52.477008 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:30:52.478587 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:30:52.479347 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:30:52.481216 (+ 1869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:30:52.481768 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:30:52.481931 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:30:52.485980 (+ 4049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:30:52.485985 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:30:52.485989 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:30:52.486028 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:30:52.489046 (+ 3018us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:30:52.489948 (+ 902us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:30:52.489953 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:30:52.489957 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:30:52.490102 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:30:52.490607 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:30:52.490612 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:30:52.490617 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:30:52.490930 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:30:52.491089 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0806 09:30:52.491955 (+ 866us) server_negotiation.cc:299] Negotiation successful
0806 09:30:52.492186 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":422,"thread_start_us":309,"threads_started":1}
I20260806 09:30:52.494004 5499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:30:52.472999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:46741 (local address 127.20.113.2:59403)
0806 09:30:52.473983 (+ 984us) negotiation.cc:107] Waiting for socket to connect
0806 09:30:52.474048 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0806 09:30:52.476427 (+ 2379us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:30:52.477268 (+ 841us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:30:52.477276 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:30:52.477747 (+ 471us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:30:52.478346 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:30:52.478366 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:30:52.479540 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:30:52.479550 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:30:52.481059 (+ 1509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:30:52.481073 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:30:52.482053 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:30:52.482061 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:30:52.482305 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:30:52.483053 (+ 748us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:30:52.483077 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:30:52.485717 (+ 2640us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:30:52.489215 (+ 3498us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:30:52.489226 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:30:52.489245 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:30:52.489809 (+ 564us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:30:52.490223 (+ 414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:30:52.490228 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:30:52.490232 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:30:52.490478 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:30:52.491069 (+ 591us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:30:52.491077 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:30:52.491539 (+ 462us) client_negotiation.cc:769] Sending connection context
0806 09:30:52.491832 (+ 293us) client_negotiation.cc:240] Negotiation successful
0806 09:30:52.492543 (+ 711us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":676,"thread_start_us":402,"threads_started":1}
I20260806 09:30:52.495705 5495 heartbeater.cc:347] Connected to a master server at 127.20.113.62:46741
I20260806 09:30:52.496052 5495 heartbeater.cc:464] Registering TS with master...
I20260806 09:30:52.497004 5495 heartbeater.cc:511] Master 127.20.113.62:46741 requested a full tablet report, sending...
I20260806 09:30:52.498955 4981 ts_manager.cc:194] Registered new tserver with Master: 13d78464f14b4e67a40b713e799f1ba1 (127.20.113.2:35221)
I20260806 09:30:52.500103 4981 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.113.2@KRBTEST.COM'} at 127.20.113.2:59403
WARNING: no policy specified for kudu/127.20.113.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.113.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.113.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.3.keytab.
Entry for principal kudu/127.20.113.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.3.keytab.
I20260806 09:30:52.556407 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.113.20:40193/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.113.3:0
--local_ip_for_outbound_sockets=127.20.113.3
--webserver_interface=127.20.113.3
--webserver_port=0
--tserver_master_addrs=127.20.113.62:46741
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.3.keytab
--principal=kudu/127.20.113.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:38379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 09:30:52.767661 5301 heartbeater.cc:503] Master 127.20.113.62:46741 was elected leader, sending a full tablet report...
W20260806 09:30:52.944082 5503 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:30:52.944402 5503 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:30:52.944502 5503 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:30:52.951930 5503 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:30:52.952042 5503 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:30:52.952170 5503 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.113.3
I20260806 09:30:52.962544 5503 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:38379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.20.113.20:40193/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.3.keytab
--principal=kudu/127.20.113.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.113.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.113.3
--webserver_port=0
--tserver_master_addrs=127.20.113.62:46741
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.113.3
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:30:52.963888 5503 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:30:52.965197 5503 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:30:52.979560 5511 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:30:52.979597 5515 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:30:52.979643 5513 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:30:52.980731 5503 server_base.cc:1034] running on GCE node
I20260806 09:30:52.981419 5503 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:30:52.982441 5503 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:30:52.983691 5503 hybrid_clock.cc:648] HybridClock initialized: now 1786008652983621 us; error 63 us; skew 500 ppm
I20260806 09:30:52.984055 5503 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:30:52 dist-test-slave-45dx krb5kdc[4311](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008652, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:30:52.987658 5503 init.cc:377] Logged in from keytab as kudu/127.20.113.3@KRBTEST.COM (short username kudu)
I20260806 09:30:52.989396 5503 webserver.cc:460] Webserver started at http://127.20.113.3:43203/ using document root <none> and password file <none>
I20260806 09:30:52.990150 5503 fs_manager.cc:359] Metadata directory not provided
I20260806 09:30:52.990370 5503 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:30:52.990737 5503 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 09:30:52 dist-test-slave-45dx krb5kdc[4311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008652, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.3@KRBTEST.COM for HTTP/127.20.113.20@KRBTEST.COM
I20260806 09:30:53.006624 5503 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/minicluster-data/ts-2/data/instance:
uuid: "9de977c75edb4d8f92a93164259b362e"
format_stamp: "Formatted at 2026-08-06 09:30:52 on dist-test-slave-45dx"
server_key: "f09fc504d290e4c97b23be966e985f98"
server_key_iv: "8209c13f4c9e0b1f7ff308535bdc50cd"
server_key_version: "kuduclusterkey@0"
I20260806 09:30:53.007619 5503 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/minicluster-data/ts-2/wal/instance:
uuid: "9de977c75edb4d8f92a93164259b362e"
format_stamp: "Formatted at 2026-08-06 09:30:52 on dist-test-slave-45dx"
server_key: "f09fc504d290e4c97b23be966e985f98"
server_key_iv: "8209c13f4c9e0b1f7ff308535bdc50cd"
server_key_version: "kuduclusterkey@0"
I20260806 09:30:53.013546 5503 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260806 09:30:53.032099 5532 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:30:53.033890 5503 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 09:30:53.034075 5503 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/minicluster-data/ts-2/wal
uuid: "9de977c75edb4d8f92a93164259b362e"
format_stamp: "Formatted at 2026-08-06 09:30:52 on dist-test-slave-45dx"
server_key: "f09fc504d290e4c97b23be966e985f98"
server_key_iv: "8209c13f4c9e0b1f7ff308535bdc50cd"
server_key_version: "kuduclusterkey@0"
I20260806 09:30:53.034328 5503 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:30:53.058965 5503 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:30:53.063364 5503 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:30:53.063649 5503 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:30:53.064640 5503 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:30:53.066107 5503 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:30:53.066193 5503 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:30:53.066280 5503 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:30:53.066326 5503 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:30:53.105708 5503 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.3:34859
I20260806 09:30:53.105744 5697 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.3:34859 every 8 connection(s)
I20260806 09:30:53.107203 5503 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/minicluster-data/ts-2/data/info.pb
I20260806 09:30:53.113526 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 5503
I20260806 09:30:53.113726 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/minicluster-data/ts-2/wal/instance
I20260806 09:30:53.114348 20932 external_mini_cluster.cc:1428] Setting key dab5ef2ef8bacee3510994bc44b275b2
Aug 06 09:30:53 dist-test-slave-45dx krb5kdc[4311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008652, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.3@KRBTEST.COM for kudu/127.20.113.62@KRBTEST.COM
I20260806 09:30:53.129567 5700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:30:53.110176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.3:56689 (local address 127.20.113.62:46741)
0806 09:30:53.110686 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0806 09:30:53.110691 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:30:53.111697 (+ 1006us) server_negotiation.cc:407] Connection header received
0806 09:30:53.113894 (+ 2197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:30:53.113898 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:30:53.113995 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:30:53.114096 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:30:53.116113 (+ 2017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:30:53.117207 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:30:53.119015 (+ 1808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:30:53.119471 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:30:53.119567 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:30:53.123327 (+ 3760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:30:53.123333 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:30:53.123336 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:30:53.123370 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:30:53.126245 (+ 2875us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:30:53.127027 (+ 782us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:30:53.127031 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:30:53.127033 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:30:53.127167 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:30:53.127598 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:30:53.127602 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:30:53.127603 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:30:53.127939 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:30:53.128095 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0806 09:30:53.129025 (+ 930us) server_negotiation.cc:299] Negotiation successful
0806 09:30:53.129279 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":397,"thread_start_us":264,"threads_started":1}
I20260806 09:30:53.130995 5701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:30:53.110755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:46741 (local address 127.20.113.3:56689)
0806 09:30:53.111533 (+ 778us) negotiation.cc:107] Waiting for socket to connect
0806 09:30:53.111573 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 09:30:53.113538 (+ 1965us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:30:53.114389 (+ 851us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:30:53.114404 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:30:53.115057 (+ 653us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:30:53.115920 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:30:53.115953 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:30:53.117402 (+ 1449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:30:53.117406 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:30:53.118855 (+ 1449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:30:53.118884 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:30:53.119698 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:30:53.119704 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:30:53.119944 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:30:53.120753 (+ 809us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:30:53.120783 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:30:53.123112 (+ 2329us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:30:53.126385 (+ 3273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:30:53.126392 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:30:53.126407 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:30:53.126893 (+ 486us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:30:53.127267 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:30:53.127270 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:30:53.127273 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:30:53.127500 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:30:53.128074 (+ 574us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:30:53.128082 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:30:53.128553 (+ 471us) client_negotiation.cc:769] Sending connection context
0806 09:30:53.128929 (+ 376us) client_negotiation.cc:240] Negotiation successful
0806 09:30:53.129501 (+ 572us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":543,"thread_start_us":319,"threads_started":1}
I20260806 09:30:53.132535 5698 heartbeater.cc:347] Connected to a master server at 127.20.113.62:46741
I20260806 09:30:53.132861 5698 heartbeater.cc:464] Registering TS with master...
I20260806 09:30:53.133610 5698 heartbeater.cc:511] Master 127.20.113.62:46741 requested a full tablet report, sending...
I20260806 09:30:53.135162 4981 ts_manager.cc:194] Registered new tserver with Master: 9de977c75edb4d8f92a93164259b362e (127.20.113.3:34859)
I20260806 09:30:53.136072 4981 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.113.3@KRBTEST.COM'} at 127.20.113.3:56689
I20260806 09:30:53.142868 20932 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 09:30:53.160732 5700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:30:53.147806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42278 (local address 127.20.113.62:46741)
0806 09:30:53.147979 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0806 09:30:53.147984 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:30:53.148541 (+ 557us) server_negotiation.cc:407] Connection header received
0806 09:30:53.148684 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:30:53.148688 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:30:53.148768 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:30:53.148850 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:30:53.150342 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:30:53.151087 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:30:53.152520 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:30:53.153099 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:30:53.153198 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:30:53.155561 (+ 2363us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:30:53.155566 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:30:53.155569 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:30:53.155602 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:30:53.158417 (+ 2815us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:30:53.159011 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:30:53.159015 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:30:53.159017 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:30:53.159142 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:30:53.159456 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:30:53.159459 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:30:53.159461 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:30:53.159715 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:30:53.159916 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0806 09:30:53.160187 (+ 271us) server_negotiation.cc:299] Negotiation successful
0806 09:30:53.160448 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 09:30:53.165019 4981 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42278:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 09:30:53.172646 4981 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 09:30:53.201478 5720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:30:53.186127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.3:34859 (local address 127.0.0.1:49792)
0806 09:30:53.187028 (+ 901us) negotiation.cc:107] Waiting for socket to connect
0806 09:30:53.187096 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0806 09:30:53.187312 (+ 216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:30:53.188839 (+ 1527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:30:53.188850 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:30:53.188892 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:30:53.189784 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:30:53.189806 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:30:53.192822 (+ 3016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:30:53.192831 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:30:53.196170 (+ 3339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:30:53.196196 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:30:53.199696 (+ 3500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:30:53.199707 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:30:53.200009 (+ 302us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:30:53.200094 (+ 85us) client_negotiation.cc:769] Sending connection context
0806 09:30:53.200264 (+ 170us) client_negotiation.cc:240] Negotiation successful
0806 09:30:53.200943 (+ 679us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":564,"thread_start_us":391,"threads_started":1}
I20260806 09:30:53.202121 5721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:30:53.186409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49792 (local address 127.20.113.3:34859)
0806 09:30:53.187995 (+ 1586us) server_negotiation.cc:206] Beginning negotiation
0806 09:30:53.188004 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 09:30:53.188038 (+ 34us) server_negotiation.cc:407] Connection header received
0806 09:30:53.188147 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:30:53.188159 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:30:53.188531 (+ 372us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:30:53.190072 (+ 1541us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:30:53.190490 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:30:53.192593 (+ 2103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:30:53.196564 (+ 3971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:30:53.198696 (+ 2132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:30:53.199625 (+ 929us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:30:53.200000 (+ 375us) server_negotiation.cc:1035] Waiting for connection context
0806 09:30:53.200805 (+ 805us) server_negotiation.cc:299] Negotiation successful
0806 09:30:53.201699 (+ 894us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1398,"thread_start_us":1476,"threads_started":1}
I20260806 09:30:53.206703 5603 tablet_service.cc:1467] Processing CreateTablet for tablet ac4f958bb1d44e34aa850cd8d74611d8 (DEFAULT_TABLE table=test-table [id=9883986478cb40a99dde80a90fea192e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:30:53.208034 5722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:30:53.187362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.1:45345 (local address 127.0.0.1:47492)
0806 09:30:53.187971 (+ 609us) negotiation.cc:107] Waiting for socket to connect
0806 09:30:53.187990 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 09:30:53.188102 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:30:53.201344 (+ 13242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:30:53.201351 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:30:53.201383 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:30:53.201774 (+ 391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:30:53.201787 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:30:53.204046 (+ 2259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:30:53.204052 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:30:53.205766 (+ 1714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:30:53.205779 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:30:53.207129 (+ 1350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:30:53.207135 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:30:53.207330 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:30:53.207368 (+ 38us) client_negotiation.cc:769] Sending connection context
0806 09:30:53.207476 (+ 108us) client_negotiation.cc:240] Negotiation successful
0806 09:30:53.207614 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":485,"mutex_wait_us":1,"thread_start_us":1462,"threads_started":1}
I20260806 09:30:53.208153 5726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:30:53.188048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47492 (local address 127.20.113.1:45345)
0806 09:30:53.200491 (+ 12443us) server_negotiation.cc:206] Beginning negotiation
0806 09:30:53.200501 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 09:30:53.200533 (+ 32us) server_negotiation.cc:407] Connection header received
0806 09:30:53.200685 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:30:53.200693 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:30:53.200986 (+ 293us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:30:53.201303 (+ 317us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:30:53.202021 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:30:53.203875 (+ 1854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:30:53.205937 (+ 2062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:30:53.206874 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:30:53.207046 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:30:53.207285 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0806 09:30:53.207546 (+ 261us) server_negotiation.cc:299] Negotiation successful
0806 09:30:53.207781 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11279,"thread_start_us":11079,"threads_started":1}
I20260806 09:30:53.208770 5603 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ac4f958bb1d44e34aa850cd8d74611d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:30:53.211086 5723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:30:53.187722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.2:35221 (local address 127.0.0.1:34394)
0806 09:30:53.201117 (+ 13395us) negotiation.cc:107] Waiting for socket to connect
0806 09:30:53.201141 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 09:30:53.201300 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:30:53.202427 (+ 1127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:30:53.202432 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:30:53.202459 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:30:53.202879 (+ 420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:30:53.202895 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:30:53.204732 (+ 1837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:30:53.204738 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:30:53.207065 (+ 2327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:30:53.207081 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:30:53.209344 (+ 2263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:30:53.209351 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:30:53.210417 (+ 1066us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:30:53.210444 (+ 27us) client_negotiation.cc:769] Sending connection context
0806 09:30:53.210557 (+ 113us) client_negotiation.cc:240] Negotiation successful
0806 09:30:53.210691 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13254,"thread_start_us":12497,"threads_started":1}
I20260806 09:30:53.211187 5727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:30:53.188524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34394 (local address 127.20.113.2:35221)
0806 09:30:53.200817 (+ 12293us) server_negotiation.cc:206] Beginning negotiation
0806 09:30:53.200823 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:30:53.201261 (+ 438us) server_negotiation.cc:407] Connection header received
0806 09:30:53.201520 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:30:53.201529 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:30:53.201820 (+ 291us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:30:53.202014 (+ 194us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:30:53.203084 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:30:53.204570 (+ 1486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:30:53.207267 (+ 2697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:30:53.208827 (+ 1560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:30:53.208985 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:30:53.209163 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0806 09:30:53.210618 (+ 1455us) server_negotiation.cc:299] Negotiation successful
0806 09:30:53.210793 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12130,"thread_start_us":12004,"threads_started":1}
I20260806 09:30:53.212365 5203 tablet_service.cc:1467] Processing CreateTablet for tablet ac4f958bb1d44e34aa850cd8d74611d8 (DEFAULT_TABLE table=test-table [id=9883986478cb40a99dde80a90fea192e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:30:53.214017 5203 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ac4f958bb1d44e34aa850cd8d74611d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:30:53.214255 5410 tablet_service.cc:1467] Processing CreateTablet for tablet ac4f958bb1d44e34aa850cd8d74611d8 (DEFAULT_TABLE table=test-table [id=9883986478cb40a99dde80a90fea192e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:30:53.215387 5410 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ac4f958bb1d44e34aa850cd8d74611d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:30:53.226691 5728 tablet_bootstrap.cc:492] T ac4f958bb1d44e34aa850cd8d74611d8 P 9de977c75edb4d8f92a93164259b362e: Bootstrap starting.
I20260806 09:30:53.227240 5730 tablet_bootstrap.cc:492] T ac4f958bb1d44e34aa850cd8d74611d8 P 13d78464f14b4e67a40b713e799f1ba1: Bootstrap starting.
I20260806 09:30:53.229671 5731 tablet_bootstrap.cc:492] T ac4f958bb1d44e34aa850cd8d74611d8 P 4c258f17206b468a9e0572cb7d716049: Bootstrap starting.
I20260806 09:30:53.231122 5730 tablet_bootstrap.cc:654] T ac4f958bb1d44e34aa850cd8d74611d8 P 13d78464f14b4e67a40b713e799f1ba1: Neither blocks nor log segments found. Creating new log.
I20260806 09:30:53.231269 5728 tablet_bootstrap.cc:654] T ac4f958bb1d44e34aa850cd8d74611d8 P 9de977c75edb4d8f92a93164259b362e: Neither blocks nor log segments found. Creating new log.
I20260806 09:30:53.232281 5730 log.cc:824] T ac4f958bb1d44e34aa850cd8d74611d8 P 13d78464f14b4e67a40b713e799f1ba1: Log is configured to *not* fsync() on all Append() calls
I20260806 09:30:53.232673 5728 log.cc:824] T ac4f958bb1d44e34aa850cd8d74611d8 P 9de977c75edb4d8f92a93164259b362e: Log is configured to *not* fsync() on all Append() calls
I20260806 09:30:53.233976 5731 tablet_bootstrap.cc:654] T ac4f958bb1d44e34aa850cd8d74611d8 P 4c258f17206b468a9e0572cb7d716049: Neither blocks nor log segments found. Creating new log.
I20260806 09:30:53.235109 5731 log.cc:824] T ac4f958bb1d44e34aa850cd8d74611d8 P 4c258f17206b468a9e0572cb7d716049: Log is configured to *not* fsync() on all Append() calls
I20260806 09:30:53.236323 5730 tablet_bootstrap.cc:492] T ac4f958bb1d44e34aa850cd8d74611d8 P 13d78464f14b4e67a40b713e799f1ba1: No bootstrap required, opened a new log
I20260806 09:30:53.236322 5728 tablet_bootstrap.cc:492] T ac4f958bb1d44e34aa850cd8d74611d8 P 9de977c75edb4d8f92a93164259b362e: No bootstrap required, opened a new log
I20260806 09:30:53.236635 5728 ts_tablet_manager.cc:1397] T ac4f958bb1d44e34aa850cd8d74611d8 P 9de977c75edb4d8f92a93164259b362e: Time spent bootstrapping tablet: real 0.010s user 0.009s sys 0.000s
I20260806 09:30:53.236624 5730 ts_tablet_manager.cc:1397] T ac4f958bb1d44e34aa850cd8d74611d8 P 13d78464f14b4e67a40b713e799f1ba1: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.004s
I20260806 09:30:53.238698 5731 tablet_bootstrap.cc:492] T ac4f958bb1d44e34aa850cd8d74611d8 P 4c258f17206b468a9e0572cb7d716049: No bootstrap required, opened a new log
I20260806 09:30:53.238988 5731 ts_tablet_manager.cc:1397] T ac4f958bb1d44e34aa850cd8d74611d8 P 4c258f17206b468a9e0572cb7d716049: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.004s
I20260806 09:30:53.247440 5731 raft_consensus.cc:348] T ac4f958bb1d44e34aa850cd8d74611d8 P 4c258f17206b468a9e0572cb7d716049 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9de977c75edb4d8f92a93164259b362e" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 34859 } } peers { permanent_uuid: "4c258f17206b468a9e0572cb7d716049" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 45345 } } peers { permanent_uuid: "13d78464f14b4e67a40b713e799f1ba1" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 35221 } }
I20260806 09:30:53.247850 5731 raft_consensus.cc:374] T ac4f958bb1d44e34aa850cd8d74611d8 P 4c258f17206b468a9e0572cb7d716049 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:30:53.247972 5731 raft_consensus.cc:729] T ac4f958bb1d44e34aa850cd8d74611d8 P 4c258f17206b468a9e0572cb7d716049 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c258f17206b468a9e0572cb7d716049, State: Initialized, Role: FOLLOWER
I20260806 09:30:53.247952 5730 raft_consensus.cc:348] T ac4f958bb1d44e34aa850cd8d74611d8 P 13d78464f14b4e67a40b713e799f1ba1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9de977c75edb4d8f92a93164259b362e" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 34859 } } peers { permanent_uuid: "4c258f17206b468a9e0572cb7d716049" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 45345 } } peers { permanent_uuid: "13d78464f14b4e67a40b713e799f1ba1" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 35221 } }
I20260806 09:30:53.248065 5728 raft_consensus.cc:348] T ac4f958bb1d44e34aa850cd8d74611d8 P 9de977c75edb4d8f92a93164259b362e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9de977c75edb4d8f92a93164259b362e" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 34859 } } peers { permanent_uuid: "4c258f17206b468a9e0572cb7d716049" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 45345 } } peers { permanent_uuid: "13d78464f14b4e67a40b713e799f1ba1" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 35221 } }
I20260806 09:30:53.248490 5730 raft_consensus.cc:374] T ac4f958bb1d44e34aa850cd8d74611d8 P 13d78464f14b4e67a40b713e799f1ba1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:30:53.248564 5728 raft_consensus.cc:374] T ac4f958bb1d44e34aa850cd8d74611d8 P 9de977c75edb4d8f92a93164259b362e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:30:53.248615 5730 raft_consensus.cc:729] T ac4f958bb1d44e34aa850cd8d74611d8 P 13d78464f14b4e67a40b713e799f1ba1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13d78464f14b4e67a40b713e799f1ba1, State: Initialized, Role: FOLLOWER
I20260806 09:30:53.248710 5728 raft_consensus.cc:729] T ac4f958bb1d44e34aa850cd8d74611d8 P 9de977c75edb4d8f92a93164259b362e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9de977c75edb4d8f92a93164259b362e, State: Initialized, Role: FOLLOWER
I20260806 09:30:53.248764 5731 consensus_queue.cc:260] T ac4f958bb1d44e34aa850cd8d74611d8 P 4c258f17206b468a9e0572cb7d716049 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9de977c75edb4d8f92a93164259b362e" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 34859 } } peers { permanent_uuid: "4c258f17206b468a9e0572cb7d716049" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 45345 } } peers { permanent_uuid: "13d78464f14b4e67a40b713e799f1ba1" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 35221 } }
I20260806 09:30:53.249265 5730 consensus_queue.cc:260] T ac4f958bb1d44e34aa850cd8d74611d8 P 13d78464f14b4e67a40b713e799f1ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9de977c75edb4d8f92a93164259b362e" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 34859 } } peers { permanent_uuid: "4c258f17206b468a9e0572cb7d716049" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 45345 } } peers { permanent_uuid: "13d78464f14b4e67a40b713e799f1ba1" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 35221 } }
I20260806 09:30:53.249349 5728 consensus_queue.cc:260] T ac4f958bb1d44e34aa850cd8d74611d8 P 9de977c75edb4d8f92a93164259b362e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9de977c75edb4d8f92a93164259b362e" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 34859 } } peers { permanent_uuid: "4c258f17206b468a9e0572cb7d716049" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 45345 } } peers { permanent_uuid: "13d78464f14b4e67a40b713e799f1ba1" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 35221 } }
I20260806 09:30:53.252491 5731 ts_tablet_manager.cc:1428] T ac4f958bb1d44e34aa850cd8d74611d8 P 4c258f17206b468a9e0572cb7d716049: Time spent starting tablet: real 0.013s user 0.008s sys 0.005s
I20260806 09:30:53.253600 5495 heartbeater.cc:503] Master 127.20.113.62:46741 was elected leader, sending a full tablet report...
I20260806 09:30:53.254289 5730 ts_tablet_manager.cc:1428] T ac4f958bb1d44e34aa850cd8d74611d8 P 13d78464f14b4e67a40b713e799f1ba1: Time spent starting tablet: real 0.018s user 0.013s sys 0.004s
I20260806 09:30:53.254320 5728 ts_tablet_manager.cc:1428] T ac4f958bb1d44e34aa850cd8d74611d8 P 9de977c75edb4d8f92a93164259b362e: Time spent starting tablet: real 0.018s user 0.013s sys 0.003s
I20260806 09:30:53.254634 5698 heartbeater.cc:503] Master 127.20.113.62:46741 was elected leader, sending a full tablet report...
W20260806 09:30:53.359139 5699 tablet.cc:2369] T ac4f958bb1d44e34aa850cd8d74611d8 P 9de977c75edb4d8f92a93164259b362e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 09:30:53.460283 5302 tablet.cc:2369] T ac4f958bb1d44e34aa850cd8d74611d8 P 4c258f17206b468a9e0572cb7d716049: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 09:30:53.471750 5496 tablet.cc:2369] T ac4f958bb1d44e34aa850cd8d74611d8 P 13d78464f14b4e67a40b713e799f1ba1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 09:30:53.499112 5744 raft_consensus.cc:482] T ac4f958bb1d44e34aa850cd8d74611d8 P 9de977c75edb4d8f92a93164259b362e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 09:30:53.499486 5744 raft_consensus.cc:504] T ac4f958bb1d44e34aa850cd8d74611d8 P 9de977c75edb4d8f92a93164259b362e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9de977c75edb4d8f92a93164259b362e" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 34859 } } peers { permanent_uuid: "4c258f17206b468a9e0572cb7d716049" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 45345 } } peers { permanent_uuid: "13d78464f14b4e67a40b713e799f1ba1" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 35221 } }
I20260806 09:30:53.501777 5744 leader_election.cc:288] T ac4f958bb1d44e34aa850cd8d74611d8 P 9de977c75edb4d8f92a93164259b362e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4c258f17206b468a9e0572cb7d716049 (127.20.113.1:45345), 13d78464f14b4e67a40b713e799f1ba1 (127.20.113.2:35221)
I20260806 09:30:53.507926 5726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:30:53.502231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.3:47925 (local address 127.20.113.1:45345)
0806 09:30:53.502562 (+ 331us) server_negotiation.cc:206] Beginning negotiation
0806 09:30:53.502568 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:30:53.502831 (+ 263us) server_negotiation.cc:407] Connection header received
0806 09:30:53.503079 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:30:53.503084 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:30:53.503145 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:30:53.503246 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:30:53.503651 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:30:53.504440 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:30:53.506168 (+ 1728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:30:53.506982 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:30:53.507165 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:30:53.507244 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 09:30:53.507519 (+ 275us) server_negotiation.cc:299] Negotiation successful
0806 09:30:53.507648 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167}
I20260806 09:30:53.507931 5701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:30:53.502153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.1:45345 (local address 127.20.113.3:47925)
0806 09:30:53.502718 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0806 09:30:53.502739 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 09:30:53.502917 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:30:53.503266 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:30:53.503271 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:30:53.503297 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:30:53.503500 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:30:53.503509 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:30:53.504570 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:30:53.504578 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:30:53.505945 (+ 1367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:30:53.505957 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:30:53.507143 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:30:53.507149 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:30:53.507340 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:30:53.507366 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 09:30:53.507493 (+ 127us) client_negotiation.cc:240] Negotiation successful
0806 09:30:53.507646 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":443}
I20260806 09:30:53.508252 5727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:30:53.502365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.3:55635 (local address 127.20.113.2:35221)
0806 09:30:53.502580 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0806 09:30:53.502588 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 09:30:53.503075 (+ 487us) server_negotiation.cc:407] Connection header received
0806 09:30:53.503273 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:30:53.503278 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:30:53.503337 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:30:53.503435 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:30:53.503995 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:30:53.504772 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:30:53.506553 (+ 1781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:30:53.507324 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:30:53.507517 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:30:53.507590 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 09:30:53.507852 (+ 262us) server_negotiation.cc:299] Negotiation successful
0806 09:30:53.507991 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 09:30:53.508395 5745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:30:53.502157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.2:35221 (local address 127.20.113.3:55635)
0806 09:30:53.502961 (+ 804us) negotiation.cc:107] Waiting for socket to connect
0806 09:30:53.502975 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 09:30:53.503123 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:30:53.503460 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:30:53.503464 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:30:53.503490 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:30:53.503835 (+ 345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:30:53.503843 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:30:53.504928 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:30:53.504933 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:30:53.506385 (+ 1452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:30:53.506395 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:30:53.507537 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:30:53.507544 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:30:53.507701 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:30:53.507723 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 09:30:53.507825 (+ 102us) client_negotiation.cc:240] Negotiation successful
0806 09:30:53.507958 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":700,"mutex_wait_us":66,"thread_start_us":403,"threads_started":1}
I20260806 09:30:53.509435 5234 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac4f958bb1d44e34aa850cd8d74611d8" candidate_uuid: "9de977c75edb4d8f92a93164259b362e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c258f17206b468a9e0572cb7d716049" is_pre_election: true
I20260806 09:30:53.509529 5440 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac4f958bb1d44e34aa850cd8d74611d8" candidate_uuid: "9de977c75edb4d8f92a93164259b362e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13d78464f14b4e67a40b713e799f1ba1" is_pre_election: true
I20260806 09:30:53.510109 5440 raft_consensus.cc:2454] T ac4f958bb1d44e34aa850cd8d74611d8 P 13d78464f14b4e67a40b713e799f1ba1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9de977c75edb4d8f92a93164259b362e in term 0.
I20260806 09:30:53.510109 5234 raft_consensus.cc:2454] T ac4f958bb1d44e34aa850cd8d74611d8 P 4c258f17206b468a9e0572cb7d716049 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9de977c75edb4d8f92a93164259b362e in term 0.
I20260806 09:30:53.510900 5537 leader_election.cc:302] T ac4f958bb1d44e34aa850cd8d74611d8 P 9de977c75edb4d8f92a93164259b362e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13d78464f14b4e67a40b713e799f1ba1, 9de977c75edb4d8f92a93164259b362e; no voters:
I20260806 09:30:53.511337 5744 raft_consensus.cc:2781] T ac4f958bb1d44e34aa850cd8d74611d8 P 9de977c75edb4d8f92a93164259b362e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 09:30:53.511515 5744 raft_consensus.cc:482] T ac4f958bb1d44e34aa850cd8d74611d8 P 9de977c75edb4d8f92a93164259b362e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 09:30:53.511629 5744 raft_consensus.cc:3037] T ac4f958bb1d44e34aa850cd8d74611d8 P 9de977c75edb4d8f92a93164259b362e [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:30:53.513983 5744 raft_consensus.cc:504] T ac4f958bb1d44e34aa850cd8d74611d8 P 9de977c75edb4d8f92a93164259b362e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9de977c75edb4d8f92a93164259b362e" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 34859 } } peers { permanent_uuid: "4c258f17206b468a9e0572cb7d716049" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 45345 } } peers { permanent_uuid: "13d78464f14b4e67a40b713e799f1ba1" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 35221 } }
I20260806 09:30:53.514633 5744 leader_election.cc:288] T ac4f958bb1d44e34aa850cd8d74611d8 P 9de977c75edb4d8f92a93164259b362e [CANDIDATE]: Term 1 election: Requested vote from peers 4c258f17206b468a9e0572cb7d716049 (127.20.113.1:45345), 13d78464f14b4e67a40b713e799f1ba1 (127.20.113.2:35221)
I20260806 09:30:53.515148 5234 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac4f958bb1d44e34aa850cd8d74611d8" candidate_uuid: "9de977c75edb4d8f92a93164259b362e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c258f17206b468a9e0572cb7d716049"
I20260806 09:30:53.515220 5440 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac4f958bb1d44e34aa850cd8d74611d8" candidate_uuid: "9de977c75edb4d8f92a93164259b362e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13d78464f14b4e67a40b713e799f1ba1"
I20260806 09:30:53.515414 5234 raft_consensus.cc:3037] T ac4f958bb1d44e34aa850cd8d74611d8 P 4c258f17206b468a9e0572cb7d716049 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:30:53.515504 5440 raft_consensus.cc:3037] T ac4f958bb1d44e34aa850cd8d74611d8 P 13d78464f14b4e67a40b713e799f1ba1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:30:53.517827 5440 raft_consensus.cc:2454] T ac4f958bb1d44e34aa850cd8d74611d8 P 13d78464f14b4e67a40b713e799f1ba1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9de977c75edb4d8f92a93164259b362e in term 1.
I20260806 09:30:53.517827 5234 raft_consensus.cc:2454] T ac4f958bb1d44e34aa850cd8d74611d8 P 4c258f17206b468a9e0572cb7d716049 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9de977c75edb4d8f92a93164259b362e in term 1.
I20260806 09:30:53.518273 5537 leader_election.cc:302] T ac4f958bb1d44e34aa850cd8d74611d8 P 9de977c75edb4d8f92a93164259b362e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13d78464f14b4e67a40b713e799f1ba1, 9de977c75edb4d8f92a93164259b362e; no voters:
I20260806 09:30:53.518603 5744 raft_consensus.cc:2781] T ac4f958bb1d44e34aa850cd8d74611d8 P 9de977c75edb4d8f92a93164259b362e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:30:53.519081 5744 raft_consensus.cc:686] T ac4f958bb1d44e34aa850cd8d74611d8 P 9de977c75edb4d8f92a93164259b362e [term 1 LEADER]: Becoming Leader. State: Replica: 9de977c75edb4d8f92a93164259b362e, State: Running, Role: LEADER
I20260806 09:30:53.519640 5744 consensus_queue.cc:237] T ac4f958bb1d44e34aa850cd8d74611d8 P 9de977c75edb4d8f92a93164259b362e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9de977c75edb4d8f92a93164259b362e" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 34859 } } peers { permanent_uuid: "4c258f17206b468a9e0572cb7d716049" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 45345 } } peers { permanent_uuid: "13d78464f14b4e67a40b713e799f1ba1" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 35221 } }
I20260806 09:30:53.524389 4980 catalog_manager.cc:5495] T ac4f958bb1d44e34aa850cd8d74611d8 P 9de977c75edb4d8f92a93164259b362e reported cstate change: term changed from 0 to 1, leader changed from <none> to 9de977c75edb4d8f92a93164259b362e (127.20.113.3). New cstate: current_term: 1 leader_uuid: "9de977c75edb4d8f92a93164259b362e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9de977c75edb4d8f92a93164259b362e" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 34859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4c258f17206b468a9e0572cb7d716049" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 45345 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13d78464f14b4e67a40b713e799f1ba1" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 35221 } health_report { overall_health: UNKNOWN } } }
I20260806 09:30:53.566147 5721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:30:53.560272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49800 (local address 127.20.113.3:34859)
0806 09:30:53.560480 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0806 09:30:53.560485 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:30:53.560621 (+ 136us) server_negotiation.cc:407] Connection header received
0806 09:30:53.560840 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:30:53.560844 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:30:53.560943 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:30:53.561078 (+ 135us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 09:30:53.561520 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:30:53.562294 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:30:53.563672 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:30:53.564073 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:30:53.564163 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:30:53.564790 (+ 627us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 09:30:53.565550 (+ 760us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 09:30:53.565746 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0806 09:30:53.565880 (+ 134us) server_negotiation.cc:299] Negotiation successful
0806 09:30:53.565947 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260806 09:30:53.574236 5440 raft_consensus.cc:1261] T ac4f958bb1d44e34aa850cd8d74611d8 P 13d78464f14b4e67a40b713e799f1ba1 [term 1 FOLLOWER]: Refusing update from remote peer 9de977c75edb4d8f92a93164259b362e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 09:30:53.574306 5234 raft_consensus.cc:1261] T ac4f958bb1d44e34aa850cd8d74611d8 P 4c258f17206b468a9e0572cb7d716049 [term 1 FOLLOWER]: Refusing update from remote peer 9de977c75edb4d8f92a93164259b362e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 09:30:53.575294 5746 consensus_queue.cc:1035] T ac4f958bb1d44e34aa850cd8d74611d8 P 9de977c75edb4d8f92a93164259b362e [LEADER]: Connected to new peer: Peer: permanent_uuid: "13d78464f14b4e67a40b713e799f1ba1" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 35221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 09:30:53.575650 5744 consensus_queue.cc:1035] T ac4f958bb1d44e34aa850cd8d74611d8 P 9de977c75edb4d8f92a93164259b362e [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c258f17206b468a9e0572cb7d716049" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 45345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 09:30:53.588234 5749 mvcc.cc:205] Tried to move back new op lower bound from 7315491445029208064 to 7315491444820594688. Current Snapshot: MvccSnapshot[applied={T|T < 7315491445029208064}]
I20260806 09:30:53.608395 4981 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42278:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 09:30:53.608716 4981 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42278:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 09:30:53.612622 4981 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4334efcb0e8a4e9cb41de47b21ce6f4e: Sending DeleteTablet for 3 replicas of tablet ac4f958bb1d44e34aa850cd8d74611d8
I20260806 09:30:53.613792 5603 tablet_service.cc:1514] Processing DeleteTablet for tablet ac4f958bb1d44e34aa850cd8d74611d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:30:53 UTC) from {username='kudu', principal='kudu/127.20.113.62@KRBTEST.COM'} at 127.0.0.1:49792
I20260806 09:30:53.613993 5203 tablet_service.cc:1514] Processing DeleteTablet for tablet ac4f958bb1d44e34aa850cd8d74611d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:30:53 UTC) from {username='kudu', principal='kudu/127.20.113.62@KRBTEST.COM'} at 127.0.0.1:47492
I20260806 09:30:53.614195 5410 tablet_service.cc:1514] Processing DeleteTablet for tablet ac4f958bb1d44e34aa850cd8d74611d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:30:53 UTC) from {username='kudu', principal='kudu/127.20.113.62@KRBTEST.COM'} at 127.0.0.1:34394
I20260806 09:30:53.615015 5762 tablet_replica.cc:330] T ac4f958bb1d44e34aa850cd8d74611d8 P 9de977c75edb4d8f92a93164259b362e: stopping tablet replica
I20260806 09:30:53.615617 5762 raft_consensus.cc:2229] T ac4f958bb1d44e34aa850cd8d74611d8 P 9de977c75edb4d8f92a93164259b362e [term 1 LEADER]: Raft consensus shutting down.
I20260806 09:30:53.616614 5762 raft_consensus.cc:2258] T ac4f958bb1d44e34aa850cd8d74611d8 P 9de977c75edb4d8f92a93164259b362e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 09:30:53.617049 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 5093
I20260806 09:30:53.622445 5762 ts_tablet_manager.cc:1905] T ac4f958bb1d44e34aa850cd8d74611d8 P 9de977c75edb4d8f92a93164259b362e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 09:30:53.631165 5762 ts_tablet_manager.cc:1918] T ac4f958bb1d44e34aa850cd8d74611d8 P 9de977c75edb4d8f92a93164259b362e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 09:30:53.632863 5763 tablet_replica.cc:330] T ac4f958bb1d44e34aa850cd8d74611d8 P 4c258f17206b468a9e0572cb7d716049: stopping tablet replica
I20260806 09:30:53.633410 5764 tablet_replica.cc:330] T ac4f958bb1d44e34aa850cd8d74611d8 P 13d78464f14b4e67a40b713e799f1ba1: stopping tablet replica
I20260806 09:30:53.633412 5763 raft_consensus.cc:2229] T ac4f958bb1d44e34aa850cd8d74611d8 P 4c258f17206b468a9e0572cb7d716049 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 09:30:53.633837 5764 raft_consensus.cc:2229] T ac4f958bb1d44e34aa850cd8d74611d8 P 13d78464f14b4e67a40b713e799f1ba1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 09:30:53.634003 5763 raft_consensus.cc:2258] T ac4f958bb1d44e34aa850cd8d74611d8 P 4c258f17206b468a9e0572cb7d716049 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 09:30:53.634361 5764 raft_consensus.cc:2258] T ac4f958bb1d44e34aa850cd8d74611d8 P 13d78464f14b4e67a40b713e799f1ba1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 09:30:53.634469 5762 log.cc:1196] T ac4f958bb1d44e34aa850cd8d74611d8 P 9de977c75edb4d8f92a93164259b362e: Deleting WAL directory at /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/minicluster-data/ts-2/wal/wals/ac4f958bb1d44e34aa850cd8d74611d8
I20260806 09:30:53.635154 5762 ts_tablet_manager.cc:1939] T ac4f958bb1d44e34aa850cd8d74611d8 P 9de977c75edb4d8f92a93164259b362e: Deleting consensus metadata
I20260806 09:30:53.636926 5764 ts_tablet_manager.cc:1905] T ac4f958bb1d44e34aa850cd8d74611d8 P 13d78464f14b4e67a40b713e799f1ba1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 09:30:53.638201 5763 ts_tablet_manager.cc:1905] T ac4f958bb1d44e34aa850cd8d74611d8 P 4c258f17206b468a9e0572cb7d716049: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 09:30:53.639293 4959 catalog_manager.cc:4841] TS 9de977c75edb4d8f92a93164259b362e (127.20.113.3:34859): tablet ac4f958bb1d44e34aa850cd8d74611d8 (table test-table [id=9883986478cb40a99dde80a90fea192e]) successfully deleted
Aug 06 09:30:53 dist-test-slave-45dx krb5kdc[4311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008649, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.113.1@KRBTEST.COM
I20260806 09:30:53.644538 5764 ts_tablet_manager.cc:1918] T ac4f958bb1d44e34aa850cd8d74611d8 P 13d78464f14b4e67a40b713e799f1ba1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 09:30:53.644754 5764 log.cc:1196] T ac4f958bb1d44e34aa850cd8d74611d8 P 13d78464f14b4e67a40b713e799f1ba1: Deleting WAL directory at /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/minicluster-data/ts-1/wal/wals/ac4f958bb1d44e34aa850cd8d74611d8
I20260806 09:30:53.645344 5764 ts_tablet_manager.cc:1939] T ac4f958bb1d44e34aa850cd8d74611d8 P 13d78464f14b4e67a40b713e799f1ba1: Deleting consensus metadata
I20260806 09:30:53.647471 4963 catalog_manager.cc:4841] TS 13d78464f14b4e67a40b713e799f1ba1 (127.20.113.2:35221): tablet ac4f958bb1d44e34aa850cd8d74611d8 (table test-table [id=9883986478cb40a99dde80a90fea192e]) successfully deleted
I20260806 09:30:53.647764 5763 ts_tablet_manager.cc:1918] T ac4f958bb1d44e34aa850cd8d74611d8 P 4c258f17206b468a9e0572cb7d716049: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 09:30:53.648012 5763 log.cc:1196] T ac4f958bb1d44e34aa850cd8d74611d8 P 4c258f17206b468a9e0572cb7d716049: Deleting WAL directory at /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786008495763054-20932-0/minicluster-data/ts-0/wal/wals/ac4f958bb1d44e34aa850cd8d74611d8
I20260806 09:30:53.648635 5763 ts_tablet_manager.cc:1939] T ac4f958bb1d44e34aa850cd8d74611d8 P 4c258f17206b468a9e0572cb7d716049: Deleting consensus metadata
I20260806 09:30:53.650599 4959 catalog_manager.cc:4841] TS 4c258f17206b468a9e0572cb7d716049 (127.20.113.1:45345): tablet ac4f958bb1d44e34aa850cd8d74611d8 (table test-table [id=9883986478cb40a99dde80a90fea192e]) successfully deleted
I20260806 09:30:53.651665 5726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:30:53.618694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47496 (local address 127.20.113.1:45345)
0806 09:30:53.618887 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0806 09:30:53.618893 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:30:53.632005 (+ 13112us) server_negotiation.cc:407] Connection header received
0806 09:30:53.632227 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:30:53.632237 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:30:53.632389 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:30:53.632541 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:30:53.634161 (+ 1620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:30:53.635174 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:30:53.637247 (+ 2073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:30:53.637863 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:30:53.637976 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:30:53.643891 (+ 5915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:30:53.643919 (+ 28us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:30:53.643937 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:30:53.643996 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:30:53.648636 (+ 4640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:30:53.649415 (+ 779us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:30:53.649423 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:30:53.649425 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:30:53.649565 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:30:53.650006 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:30:53.650013 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:30:53.650021 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:30:53.650487 (+ 466us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:30:53.650812 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0806 09:30:53.651151 (+ 339us) server_negotiation.cc:299] Negotiation successful
0806 09:30:53.651311 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 09:30:53.652592 5295 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:30:53.744002 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 5093
I20260806 09:30:53.777621 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 5308
Aug 06 09:30:53 dist-test-slave-45dx krb5kdc[4311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008649, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.113.2@KRBTEST.COM
I20260806 09:30:53.795792 5727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:30:53.778504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34404 (local address 127.20.113.2:35221)
0806 09:30:53.778711 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0806 09:30:53.778716 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:30:53.778811 (+ 95us) server_negotiation.cc:407] Connection header received
0806 09:30:53.779006 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:30:53.779011 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:30:53.779078 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:30:53.779163 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:30:53.780826 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:30:53.781506 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:30:53.783442 (+ 1936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:30:53.783832 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:30:53.783929 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:30:53.789483 (+ 5554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:30:53.789500 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:30:53.789517 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:30:53.789566 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:30:53.792809 (+ 3243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:30:53.793718 (+ 909us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:30:53.793726 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:30:53.793728 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:30:53.793859 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:30:53.794283 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:30:53.794287 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:30:53.794289 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:30:53.794677 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:30:53.794947 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0806 09:30:53.795264 (+ 317us) server_negotiation.cc:299] Negotiation successful
0806 09:30:53.795438 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 09:30:53.796694 5489 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:30:53.898003 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 5308
I20260806 09:30:53.923573 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 5503
Aug 06 09:30:53 dist-test-slave-45dx krb5kdc[4311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008649, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.113.3@KRBTEST.COM
I20260806 09:30:53.940755 5721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:30:53.924451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49802 (local address 127.20.113.3:34859)
0806 09:30:53.924651 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0806 09:30:53.924661 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 09:30:53.924751 (+ 90us) server_negotiation.cc:407] Connection header received
0806 09:30:53.924998 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:30:53.925002 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:30:53.925083 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:30:53.925203 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:30:53.926579 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:30:53.927408 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:30:53.928957 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:30:53.929445 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:30:53.929598 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:30:53.934142 (+ 4544us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:30:53.934155 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:30:53.934172 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:30:53.934212 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:30:53.937454 (+ 3242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:30:53.938272 (+ 818us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:30:53.938279 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:30:53.938282 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:30:53.938460 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:30:53.938875 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:30:53.938884 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:30:53.938886 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:30:53.939296 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:30:53.939709 (+ 413us) server_negotiation.cc:1035] Waiting for connection context
0806 09:30:53.940075 (+ 366us) server_negotiation.cc:299] Negotiation successful
0806 09:30:53.940364 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 09:30:53.941550 5692 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:30:54.064977 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 5503
I20260806 09:30:54.094146 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 4938
I20260806 09:30:54.095409 5023 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:30:54.207873 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 4938
I20260806 09:30:54.238631 20932 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260806 09:30:54.584952 20932 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260806 09:30:54.585278 20932 mini_ranger.cc:66] Stopping Ranger...
I20260806 09:30:54.932123 20932 mini_ranger.cc:68] Stopped Ranger
2026-08-06 09:30:54.932 UTC [4349] LOG: received smart shutdown request
2026-08-06 09:30:54.936 UTC [4349] LOG: background worker "logical replication launcher" (PID 4357) exited with exit code 1
2026-08-06 09:30:54.937 UTC [4352] LOG: shutting down
2026-08-06 09:30:55.093 UTC [4349] LOG: database system is shut down
2026-08-06T09:30:55Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (51967 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 09:30:55 dist-test-slave-45dx krb5kdc[5780](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 09:30:55 dist-test-slave-45dx krb5kdc[5780](info): set up 2 sockets
Aug 06 09:30:55 dist-test-slave-45dx krb5kdc[5780](info): commencing operation
krb5kdc: starting...
W20260806 09:30:56.541473 20932 mini_kdc.cc:121] Time spent starting KDC: real 1.279s user 0.002s sys 0.051s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/krb5kdc/test-user.keytab.
Aug 06 09:30:56 dist-test-slave-45dx krb5kdc[5780](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008656, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T09:30:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T09:30:56Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.20.113.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.20.113.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.20.113.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/krb5kdc/rangeradmin_127.20.113.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.20.113.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/krb5kdc/rangeradmin_127.20.113.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.20.113.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.20.113.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.20.113.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/krb5kdc/rangerlookup_127.20.113.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.20.113.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/krb5kdc/rangerlookup_127.20.113.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.20.113.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.20.113.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.113.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/krb5kdc/HTTP_127.20.113.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.20.113.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/krb5kdc/HTTP_127.20.113.20@KRBTEST.COM.keytab.
I20260806 09:30:56.971760 20932 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskKR4MEC/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/postgres -l logfile start
2026-08-06 09:30:59.419 UTC [5820] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-06 09:30:59.419 UTC [5820] LOG: listening on IPv4 address "127.20.113.20", port 42041
2026-08-06 09:30:59.422 UTC [5820] LOG: listening on Unix socket "/tmp/.s.PGSQL.42041"
2026-08-06 09:30:59.445 UTC [5823] LOG: database system was shut down at 2026-08-06 09:30:58 UTC
2026-08-06 09:30:59.449 UTC [5820] LOG: database system is ready to accept connections
I20260806 09:31:00.611475 20932 mini_postgres.cc:96] Postgres bound to 42041
2026-08-06 09:31:00.639 UTC [5832] FATAL: database "slave" does not exist
127.20.113.20:42041 - accepting connections
I20260806 09:31:00.640596 20932 mini_ranger.cc:161] Starting Ranger...
I20260806 09:31:00.673229 20932 mini_ranger.cc:84] Created miniranger Postgres user
I20260806 09:31:01.137626 20932 mini_ranger.cc:87] Created ranger Postgres database
I20260806 09:31:01.137733 20932 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ranger-admin
2026-08-06 09:31:01,417 [I] DB FLAVOR :POSTGRES
2026-08-06 09:31:01,418 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 09:31:01,418 [I] Checking connection..
2026-08-06 09:31:01,418 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKR4MEC/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKR4MEC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.113.20:42041/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-06 09:31:01,690 [I] Checking connection passed.
2026-08-06 09:31:01,690 [I] --------- Verifying version history table ---------
2026-08-06 09:31:01,690 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKR4MEC/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKR4MEC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.113.20:42041/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 09:31:01,958 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 09:31:01,959 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKR4MEC/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKR4MEC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.113.20:42041/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 09:31:02,194 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 09:31:02,194 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-06 09:31:02,195 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKR4MEC/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKR4MEC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.113.20:42041/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskKR4MEC/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-06 09:31:02.437 UTC [5920] WARNING: there is no transaction in progress
2026-08-06 09:31:02,447 [I] create_dbversion_catalog.sql file imported successfully
2026-08-06 09:31:02,447 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKR4MEC/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKR4MEC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.113.20:42041/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 09:31:02,704 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-06 09:31:02,704 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 09:31:02,704 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKR4MEC/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKR4MEC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.113.20:42041/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-06 09:31:02,959 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKR4MEC/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKR4MEC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.113.20:42041/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-06 09:31:03,222 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKR4MEC/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKR4MEC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.113.20:42041/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-45dx.c.gcp-upstream.internal','N') ;"
2026-08-06 09:31:03,479 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKR4MEC/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKR4MEC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.113.20:42041/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 09:31:03,736 [I] Table x_portal_user does not exist in database ranger
2026-08-06 09:31:03,736 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKR4MEC/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKR4MEC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.113.20:42041/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 09:31:03,986 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-06 09:31:03,987 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKR4MEC/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKR4MEC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.113.20:42041/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 09:31:04,224 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKR4MEC/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKR4MEC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.113.20:42041/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 09:31:04,492 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-06 09:31:04,492 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKR4MEC/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKR4MEC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.113.20:42041/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskKR4MEC/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-06 09:31:05.437 UTC [6061] WARNING: there is no transaction in progress
2026-08-06 09:31:05.451 UTC [6061] WARNING: there is no transaction in progress
2026-08-06 09:31:05.467 UTC [6061] WARNING: there is no transaction in progress
2026-08-06 09:31:05.482 UTC [6061] WARNING: there is no transaction in progress
2026-08-06 09:31:05.497 UTC [6061] WARNING: there is no transaction in progress
2026-08-06 09:31:05.512 UTC [6061] WARNING: there is no transaction in progress
2026-08-06 09:31:05.566 UTC [6061] WARNING: there is no transaction in progress
2026-08-06 09:31:05.573 UTC [6061] WARNING: there is no transaction in progress
2026-08-06 09:31:05.586 UTC [6061] WARNING: there is no transaction in progress
2026-08-06 09:31:05.596 UTC [6061] WARNING: there is no transaction in progress
2026-08-06 09:31:05.606 UTC [6061] WARNING: there is no transaction in progress
2026-08-06 09:31:05.619 UTC [6061] WARNING: there is no transaction in progress
2026-08-06 09:31:05.629 UTC [6061] WARNING: there is no transaction in progress
2026-08-06 09:31:05.635 UTC [6061] WARNING: there is no transaction in progress
2026-08-06 09:31:05.641 UTC [6061] WARNING: there is no transaction in progress
2026-08-06 09:31:05.648 UTC [6061] WARNING: there is no transaction in progress
2026-08-06 09:31:06,245 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-06 09:31:06,246 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKR4MEC/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKR4MEC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.113.20:42041/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-06 09:31:06,501 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-06 09:31:06,502 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKR4MEC/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKR4MEC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.113.20:42041/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 09:31:06,750 [I] Table x_portal_user already exists in database 'ranger'
2026-08-06 09:31:06,751 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKR4MEC/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKR4MEC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.113.20:42041/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 09:31:06,990 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-06 09:31:06,991 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKR4MEC/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKR4MEC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.113.20:42041/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 09:31:07,254 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKR4MEC/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKR4MEC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.113.20:42041/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 09:31:07,520 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKR4MEC/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKR4MEC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.113.20:42041/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-45dx.c.gcp-upstream.internal';"
2026-08-06 09:31:07,774 [I] CORE_DB_SCHEMA import status has been updated
2026-08-06 09:31:07,775 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKR4MEC/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKR4MEC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.113.20:42041/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-06 09:31:08,019 [I] DB_PATCHES have already been applied
I20260806 09:31:08.025758 20932 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ranger-admin:/tmp/dist-test-taskKR4MEC/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskKR4MEC/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskKR4MEC/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKR4MEC/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260806 09:31:08.025861 20932 mini_ranger.cc:193] Using host: 127.20.113.20
I20260806 09:31:08.051640 20932 mini_ranger.cc:227] Ranger admin URL: http://127.20.113.20:35159
Aug 06, 2026 9:31:08 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 9:31:08 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 9:31:08 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ranger-admin/ews/webapp
Aug 06, 2026 9:31:08 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 06, 2026 9:31:08 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 06, 2026 9:31:09 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ranger-admin/ews/webapp].
Aug 06, 2026 9:31:09 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260806 09:31:09.436266 20932 mini_ranger.cc:160] Time spent starting Ranger: real 8.796s user 0.000s sys 0.076s
Aug 06, 2026 9:31:29 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 06, 2026 9:31:29 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 06, 2026 9:31:29 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 06, 2026 9:31:29 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 06, 2026 9:31:29 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 06, 2026 9:31:29 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 06, 2026 9:31:29 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 06, 2026 9:31:29 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 06, 2026 9:31:29 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 06, 2026 9:31:29 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 06, 2026 9:31:29 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 06, 2026 9:31:29 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 06, 2026 9:31:29 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 06, 2026 9:31:29 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 06, 2026 9:31:29 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 06, 2026 9:31:29 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 06, 2026 9:31:29 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 06, 2026 9:31:30 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260806 09:31:30.763038 20932 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.20.113.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.20.113.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.20.113.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/krb5kdc/rangerkms_127.20.113.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.20.113.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/krb5kdc/rangerkms_127.20.113.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.20.113.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.20.113.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.113.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/krb5kdc/HTTP_127.20.113.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.20.113.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/krb5kdc/HTTP_127.20.113.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 06 09:31:31 dist-test-slave-45dx krb5kdc[5780](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008691, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260806 09:31:31.013346 20932 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260806 09:31:31.043411 20932 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260806 09:31:31.862123 20932 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260806 09:31:31.862248 20932 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ranger-kms
I20260806 09:31:31.862303 20932 mini_ranger_kms.cc:227] Using postgres at 127.20.113.20:42041
2026-08-06 09:31:32,253 [I] DB FLAVOR :POSTGRES
2026-08-06 09:31:32,254 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 09:31:32,254 [I] Checking connection
2026-08-06 09:31:32,254 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKR4MEC/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.113.20:42041/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-06 09:31:32,496 [I] connection success
2026-08-06 09:31:32,496 [I] --------- Verifying Ranger DB tables ---------
2026-08-06 09:31:32,497 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-06 09:31:32,497 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKR4MEC/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.113.20:42041/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-06 09:31:32,741 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-06 09:31:32,741 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 09:31:32,741 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-06 09:31:32,741 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKR4MEC/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.113.20:42041/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-06 09:31:33,001 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260806 09:31:33.461104 20932 mini_ranger_kms.cc:310] Created kms service
I20260806 09:31:33.684479 20932 mini_ranger_kms.cc:326] Created kudu user
I20260806 09:31:33.729166 20932 mini_ranger_kms.cc:343] Created rangerkms user
Aug 06 09:31:33 dist-test-slave-45dx krb5kdc[5780](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008691, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.20.113.20@KRBTEST.COM
I20260806 09:31:34.124787 20932 mini_ranger_kms.cc:384] Added ranger policy
I20260806 09:31:34.125125 20932 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ranger-kms:/tmp/dist-test-taskKR4MEC/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKR4MEC/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskKR4MEC/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260806 09:31:34.125171 20932 mini_ranger_kms.cc:240] Using host: 127.20.113.20
I20260806 09:31:34.156424 20932 mini_ranger_kms.cc:276] Ranger KMS PID: 6312
I20260806 09:31:34.156562 20932 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.20.113.20:58875
Aug 06, 2026 9:31:35 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 9:31:35 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 9:31:35 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 06, 2026 9:31:35 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 06, 2026 9:31:35 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ranger-kms/ews/webapp].
Aug 06, 2026 9:31:35 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 06 09:31:36 dist-test-slave-45dx krb5kdc[5780](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786008696, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.20.113.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:31:36.932529 20932 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.919s user 0.063s sys 0.377s
I20260806 09:31:36.932811 20932 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260806 09:31:36.933009 20932 mini_ranger_kms.cc:401] 127.20.113.20:58875/kms/v1/keys
Aug 06 09:31:37 dist-test-slave-45dx krb5kdc[5780](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008696, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.20.113.20@KRBTEST.COM for HTTP/127.20.113.20@KRBTEST.COM
Aug 06 09:31:37 dist-test-slave-45dx krb5kdc[5780](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008696, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.20.113.20@KRBTEST.COM for HTTP/127.20.113.20@KRBTEST.COM
Aug 06 09:31:39 dist-test-slave-45dx krb5kdc[5780](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008699, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.20.113.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.113.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.113.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab.
Entry for principal kudu/127.20.113.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab.
I20260806 09:31:39.947357 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.113.62:40899
--webserver_interface=127.20.113.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab
--principal=kudu/127.20.113.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:33683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.113.62:40899
--ranger_config_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.113.20:58875/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:31:40.308390 6387 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:31:40.308713 6387 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:31:40.308878 6387 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:31:40.316015 6387 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 09:31:40.316156 6387 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:31:40.316253 6387 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:31:40.316409 6387 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 09:31:40.316498 6387 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 09:31:40.326556 6387 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:33683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/master-0/wal
--ranger_kms_url=127.20.113.20:58875/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.20.113.62:40899
--ranger_config_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab
--principal=kudu/127.20.113.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.113.62:40899
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.113.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:31:40.327956 6387 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:31:40.329391 6387 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:31:40.341964 6394 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:31:40.342031 6393 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:31:40.341998 6396 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:31:40.342823 6387 server_base.cc:1034] running on GCE node
I20260806 09:31:40.343680 6387 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:31:40.345177 6387 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:31:40.346424 6387 hybrid_clock.cc:648] HybridClock initialized: now 1786008700346386 us; error 46 us; skew 500 ppm
I20260806 09:31:40.346798 6387 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:31:40 dist-test-slave-45dx krb5kdc[5780](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008700, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:31:40.355270 6387 init.cc:377] Logged in from keytab as kudu/127.20.113.62@KRBTEST.COM (short username kudu)
I20260806 09:31:40.356846 6387 webserver.cc:460] Webserver started at http://127.20.113.62:39251/ using document root <none> and password file <none>
I20260806 09:31:40.357455 6387 fs_manager.cc:359] Metadata directory not provided
I20260806 09:31:40.357568 6387 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:31:40.357842 6387 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 09:31:40 dist-test-slave-45dx krb5kdc[5780](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008700, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.62@KRBTEST.COM for HTTP/127.20.113.20@KRBTEST.COM
I20260806 09:31:40.415328 6387 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/master-0/data/instance:
uuid: "4acecc18510e46919fbd36bbdbe95ed7"
format_stamp: "Formatted at 2026-08-06 09:31:40 on dist-test-slave-45dx"
server_key: "ec777597967426e54ddc942f6624c026"
server_key_iv: "b2ae7a4c207a74654b7cb4ff44f12c05"
server_key_version: "kuduclusterkey@0"
I20260806 09:31:40.416172 6387 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/master-0/wal/instance:
uuid: "4acecc18510e46919fbd36bbdbe95ed7"
format_stamp: "Formatted at 2026-08-06 09:31:40 on dist-test-slave-45dx"
server_key: "ec777597967426e54ddc942f6624c026"
server_key_iv: "b2ae7a4c207a74654b7cb4ff44f12c05"
server_key_version: "kuduclusterkey@0"
I20260806 09:31:40.421890 6387 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 09:31:40.441378 6403 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:31:40.443042 6387 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 09:31:40.443245 6387 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/master-0/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/master-0/wal
uuid: "4acecc18510e46919fbd36bbdbe95ed7"
format_stamp: "Formatted at 2026-08-06 09:31:40 on dist-test-slave-45dx"
server_key: "ec777597967426e54ddc942f6624c026"
server_key_iv: "b2ae7a4c207a74654b7cb4ff44f12c05"
server_key_version: "kuduclusterkey@0"
I20260806 09:31:40.443471 6387 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:31:40.461261 6387 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:31:40.465528 6387 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:31:40.465874 6387 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:31:40.487038 6387 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.62:40899
I20260806 09:31:40.487038 6484 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.62:40899 every 8 connection(s)
I20260806 09:31:40.488550 6387 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
I20260806 09:31:40.492324 6486 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:31:40.493094 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 6387
I20260806 09:31:40.493325 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/master-0/wal/instance
I20260806 09:31:40.494257 20932 external_mini_cluster.cc:1428] Setting key c65d5fbdbc5e0ccf67f6be054c0eea0c
I20260806 09:31:40.505767 6486 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4acecc18510e46919fbd36bbdbe95ed7: Bootstrap starting.
Aug 06 09:31:40 dist-test-slave-45dx krb5kdc[5780](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008699, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.113.62@KRBTEST.COM
I20260806 09:31:40.509132 6486 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4acecc18510e46919fbd36bbdbe95ed7: Neither blocks nor log segments found. Creating new log.
I20260806 09:31:40.510432 6486 log.cc:824] T 00000000000000000000000000000000 P 4acecc18510e46919fbd36bbdbe95ed7: Log is configured to *not* fsync() on all Append() calls
I20260806 09:31:40.513779 6486 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4acecc18510e46919fbd36bbdbe95ed7: No bootstrap required, opened a new log
I20260806 09:31:40.517299 6489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:31:40.496513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59902 (local address 127.20.113.62:40899)
0806 09:31:40.497322 (+ 809us) server_negotiation.cc:206] Beginning negotiation
0806 09:31:40.497337 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 09:31:40.497379 (+ 42us) server_negotiation.cc:407] Connection header received
0806 09:31:40.499101 (+ 1722us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:31:40.499124 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:31:40.499546 (+ 422us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:31:40.499960 (+ 414us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:31:40.501450 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:31:40.502751 (+ 1301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:31:40.504078 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:31:40.504843 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:31:40.505074 (+ 231us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:31:40.509102 (+ 4028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:31:40.509110 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:31:40.509127 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:31:40.509165 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:31:40.512831 (+ 3666us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:31:40.513614 (+ 783us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:31:40.513621 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:31:40.513628 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:31:40.513794 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:31:40.514221 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:31:40.514225 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:31:40.514228 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:31:40.514761 (+ 533us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:31:40.515116 (+ 355us) server_negotiation.cc:1035] Waiting for connection context
0806 09:31:40.515764 (+ 648us) server_negotiation.cc:299] Negotiation successful
0806 09:31:40.516169 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":589,"thread_start_us":354,"threads_started":1}
I20260806 09:31:40.522630 6486 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4acecc18510e46919fbd36bbdbe95ed7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4acecc18510e46919fbd36bbdbe95ed7" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 40899 } }
I20260806 09:31:40.523020 6486 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4acecc18510e46919fbd36bbdbe95ed7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:31:40.523128 6486 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4acecc18510e46919fbd36bbdbe95ed7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4acecc18510e46919fbd36bbdbe95ed7, State: Initialized, Role: FOLLOWER
I20260806 09:31:40.523686 6486 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4acecc18510e46919fbd36bbdbe95ed7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4acecc18510e46919fbd36bbdbe95ed7" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 40899 } }
I20260806 09:31:40.523947 6486 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4acecc18510e46919fbd36bbdbe95ed7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 09:31:40.524091 6486 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4acecc18510e46919fbd36bbdbe95ed7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 09:31:40.524267 6486 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4acecc18510e46919fbd36bbdbe95ed7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:31:40.526772 6486 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4acecc18510e46919fbd36bbdbe95ed7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4acecc18510e46919fbd36bbdbe95ed7" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 40899 } }
I20260806 09:31:40.527281 6486 leader_election.cc:302] T 00000000000000000000000000000000 P 4acecc18510e46919fbd36bbdbe95ed7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4acecc18510e46919fbd36bbdbe95ed7; no voters:
I20260806 09:31:40.527870 6486 leader_election.cc:288] T 00000000000000000000000000000000 P 4acecc18510e46919fbd36bbdbe95ed7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 09:31:40.528023 6492 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4acecc18510e46919fbd36bbdbe95ed7 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:31:40.528824 6492 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4acecc18510e46919fbd36bbdbe95ed7 [term 1 LEADER]: Becoming Leader. State: Replica: 4acecc18510e46919fbd36bbdbe95ed7, State: Running, Role: LEADER
I20260806 09:31:40.529376 6492 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4acecc18510e46919fbd36bbdbe95ed7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4acecc18510e46919fbd36bbdbe95ed7" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 40899 } }
I20260806 09:31:40.530138 6486 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4acecc18510e46919fbd36bbdbe95ed7 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 09:31:40.532326 6493 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4acecc18510e46919fbd36bbdbe95ed7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4acecc18510e46919fbd36bbdbe95ed7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4acecc18510e46919fbd36bbdbe95ed7" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 40899 } } }
I20260806 09:31:40.532431 6496 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4acecc18510e46919fbd36bbdbe95ed7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4acecc18510e46919fbd36bbdbe95ed7. Latest consensus state: current_term: 1 leader_uuid: "4acecc18510e46919fbd36bbdbe95ed7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4acecc18510e46919fbd36bbdbe95ed7" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 40899 } } }
I20260806 09:31:40.532887 6496 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4acecc18510e46919fbd36bbdbe95ed7 [sys.catalog]: This master's current role is: LEADER
I20260806 09:31:40.532887 6493 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4acecc18510e46919fbd36bbdbe95ed7 [sys.catalog]: This master's current role is: LEADER
I20260806 09:31:40.535847 6500 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 09:31:40.542590 6500 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 09:31:40.543246 6486 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 09:31:40.563350 6500 catalog_manager.cc:1347] Generated new cluster ID: 9f7cf3db701d43c5b764a7d82e4a07b8
I20260806 09:31:40.563529 6500 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 09:31:40.573732 6500 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 09:31:40.575450 6500 catalog_manager.cc:1504] Loading token signing keys...
I20260806 09:31:40.586251 6500 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4acecc18510e46919fbd36bbdbe95ed7: Generated new TSK 0
I20260806 09:31:40.587554 6500 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.113.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.113.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.113.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.1.keytab.
Entry for principal kudu/127.20.113.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.1.keytab.
I20260806 09:31:41.365793 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.113.20:58875/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.113.1:0
--local_ip_for_outbound_sockets=127.20.113.1
--webserver_interface=127.20.113.1
--webserver_port=0
--tserver_master_addrs=127.20.113.62:40899
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.1.keytab
--principal=kudu/127.20.113.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:33683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 09:31:41 dist-test-slave-45dx krb5kdc[5780](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786008701, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260806 09:31:41.827469 6552 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:31:41.827975 6552 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:31:41.828172 6552 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:31:41.840353 6552 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:31:41.840565 6552 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:31:41.840842 6552 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.113.1
I20260806 09:31:41.856171 6552 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:33683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.20.113.20:58875/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.1.keytab
--principal=kudu/127.20.113.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.113.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.113.1
--webserver_port=0
--tserver_master_addrs=127.20.113.62:40899
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.113.1
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:31:41.858050 6552 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:31:41.859413 6552 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:31:41.875964 6565 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:31:41.879304 6566 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:31:41.880491 6568 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:31:41.880957 6552 server_base.cc:1034] running on GCE node
I20260806 09:31:41.881660 6552 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:31:41.882753 6552 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:31:41.883980 6552 hybrid_clock.cc:648] HybridClock initialized: now 1786008701883923 us; error 51 us; skew 500 ppm
I20260806 09:31:41.884351 6552 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:31:41 dist-test-slave-45dx krb5kdc[5780](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008701, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:31:41.887801 6552 init.cc:377] Logged in from keytab as kudu/127.20.113.1@KRBTEST.COM (short username kudu)
I20260806 09:31:41.889482 6552 webserver.cc:460] Webserver started at http://127.20.113.1:46393/ using document root <none> and password file <none>
I20260806 09:31:41.890218 6552 fs_manager.cc:359] Metadata directory not provided
I20260806 09:31:41.890375 6552 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:31:41.890703 6552 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 09:31:41 dist-test-slave-45dx krb5kdc[5780](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008701, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.1@KRBTEST.COM for HTTP/127.20.113.20@KRBTEST.COM
I20260806 09:31:41.909641 6552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ts-0/data/instance:
uuid: "66040518f530435183215ce1112bd084"
format_stamp: "Formatted at 2026-08-06 09:31:41 on dist-test-slave-45dx"
server_key: "3c7b5a235d1f9520a4fb2e7b22c64352"
server_key_iv: "16a826bc6d0365c0ab789d79a3b53cbc"
server_key_version: "kuduclusterkey@0"
I20260806 09:31:41.910853 6552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ts-0/wal/instance:
uuid: "66040518f530435183215ce1112bd084"
format_stamp: "Formatted at 2026-08-06 09:31:41 on dist-test-slave-45dx"
server_key: "3c7b5a235d1f9520a4fb2e7b22c64352"
server_key_iv: "16a826bc6d0365c0ab789d79a3b53cbc"
server_key_version: "kuduclusterkey@0"
I20260806 09:31:41.924819 6552 fs_manager.cc:691] Time spent creating directory manager: real 0.013s user 0.001s sys 0.011s
I20260806 09:31:41.952015 6580 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:31:41.954185 6552 fs_manager.cc:725] Time spent opening block manager: real 0.011s user 0.004s sys 0.006s
I20260806 09:31:41.954460 6552 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ts-0/wal
uuid: "66040518f530435183215ce1112bd084"
format_stamp: "Formatted at 2026-08-06 09:31:41 on dist-test-slave-45dx"
server_key: "3c7b5a235d1f9520a4fb2e7b22c64352"
server_key_iv: "16a826bc6d0365c0ab789d79a3b53cbc"
server_key_version: "kuduclusterkey@0"
I20260806 09:31:41.954769 6552 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:31:41.990360 6552 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:31:41.995857 6552 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:31:41.996224 6552 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:31:41.997505 6552 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:31:41.999312 6552 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:31:41.999420 6552 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:31:41.999603 6552 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:31:41.999683 6552 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:31:42.064261 6552 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.1:33547
I20260806 09:31:42.064320 6738 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.1:33547 every 8 connection(s)
I20260806 09:31:42.066080 6552 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ts-0/data/info.pb
I20260806 09:31:42.066573 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 6552
I20260806 09:31:42.066788 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ts-0/wal/instance
I20260806 09:31:42.067552 20932 external_mini_cluster.cc:1428] Setting key 165170097735bf0a8ed1045108ec6978
Aug 06 09:31:42 dist-test-slave-45dx krb5kdc[5780](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008701, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.1@KRBTEST.COM for kudu/127.20.113.62@KRBTEST.COM
I20260806 09:31:42.109395 6741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:31:42.071565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.1:42513 (local address 127.20.113.62:40899)
0806 09:31:42.075134 (+ 3569us) server_negotiation.cc:206] Beginning negotiation
0806 09:31:42.075144 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 09:31:42.087786 (+ 12642us) server_negotiation.cc:407] Connection header received
0806 09:31:42.089823 (+ 2037us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:31:42.089830 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:31:42.089924 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:31:42.090006 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:31:42.092274 (+ 2268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:31:42.093512 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:31:42.095818 (+ 2306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:31:42.096507 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:31:42.096608 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:31:42.101378 (+ 4770us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:31:42.101386 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:31:42.101391 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:31:42.101434 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:31:42.105370 (+ 3936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:31:42.106327 (+ 957us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:31:42.106334 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:31:42.106338 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:31:42.106521 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:31:42.107233 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:31:42.107239 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:31:42.107242 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:31:42.107613 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:31:42.107747 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0806 09:31:42.108773 (+ 1026us) server_negotiation.cc:299] Negotiation successful
0806 09:31:42.108998 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3389,"thread_start_us":3253,"threads_started":1}
I20260806 09:31:42.111793 6743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:31:42.073033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:40899 (local address 127.20.113.1:42513)
0806 09:31:42.086863 (+ 13830us) negotiation.cc:107] Waiting for socket to connect
0806 09:31:42.086941 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0806 09:31:42.089461 (+ 2520us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:31:42.090599 (+ 1138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:31:42.090612 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:31:42.091249 (+ 637us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:31:42.092067 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:31:42.092090 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:31:42.093726 (+ 1636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:31:42.093731 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:31:42.095644 (+ 1913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:31:42.095668 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:31:42.096961 (+ 1293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:31:42.096967 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:31:42.097248 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:31:42.098091 (+ 843us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:31:42.098120 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:31:42.101057 (+ 2937us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:31:42.105564 (+ 4507us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:31:42.105573 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:31:42.105592 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:31:42.106183 (+ 591us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:31:42.106828 (+ 645us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:31:42.106833 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:31:42.106837 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:31:42.107094 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:31:42.107862 (+ 768us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:31:42.107869 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:31:42.108293 (+ 424us) client_negotiation.cc:769] Sending connection context
0806 09:31:42.109887 (+ 1594us) client_negotiation.cc:240] Negotiation successful
0806 09:31:42.110445 (+ 558us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13416,"thread_start_us":13552,"threads_started":1}
I20260806 09:31:42.114109 6739 heartbeater.cc:347] Connected to a master server at 127.20.113.62:40899
I20260806 09:31:42.114456 6739 heartbeater.cc:464] Registering TS with master...
I20260806 09:31:42.115389 6739 heartbeater.cc:511] Master 127.20.113.62:40899 requested a full tablet report, sending...
Aug 06 09:31:42 dist-test-slave-45dx krb5kdc[5780](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008701, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.62@KRBTEST.COM for HTTP/127.20.113.20@KRBTEST.COM
I20260806 09:31:42.118049 6434 ts_manager.cc:194] Registered new tserver with Master: 66040518f530435183215ce1112bd084 (127.20.113.1:33547)
I20260806 09:31:42.120503 6434 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.113.1@KRBTEST.COM'} at 127.20.113.1:42513
WARNING: no policy specified for kudu/127.20.113.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.113.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.113.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.2.keytab.
Entry for principal kudu/127.20.113.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.2.keytab.
W20260806 09:31:42.187016 20932 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.20.113.2: real 0.116s user 0.001s sys 0.027s
I20260806 09:31:42.188012 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.113.20:58875/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.113.2:0
--local_ip_for_outbound_sockets=127.20.113.2
--webserver_interface=127.20.113.2
--webserver_port=0
--tserver_master_addrs=127.20.113.62:40899
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.2.keytab
--principal=kudu/127.20.113.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:33683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 09:31:42 dist-test-slave-45dx krb5kdc[5780](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008701, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.62@KRBTEST.COM for HTTP/127.20.113.20@KRBTEST.COM
W20260806 09:31:42.736676 6750 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:31:42.737116 6750 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:31:42.737295 6750 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:31:42.745306 6750 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:31:42.745461 6750 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:31:42.745654 6750 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.113.2
I20260806 09:31:42.756067 6750 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:33683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.20.113.20:58875/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.2.keytab
--principal=kudu/127.20.113.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.113.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.113.2
--webserver_port=0
--tserver_master_addrs=127.20.113.62:40899
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.113.2
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:31:42.757642 6750 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:31:42.759080 6750 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:31:42.781060 6762 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:31:42.781961 6761 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:31:42.787720 6750 server_base.cc:1034] running on GCE node
W20260806 09:31:42.788223 6766 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:31:42.789018 6750 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:31:42.790257 6750 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:31:42.791544 6750 hybrid_clock.cc:648] HybridClock initialized: now 1786008702791477 us; error 59 us; skew 500 ppm
I20260806 09:31:42.791915 6750 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:31:42 dist-test-slave-45dx krb5kdc[5780](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008702, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:31:42.795706 6750 init.cc:377] Logged in from keytab as kudu/127.20.113.2@KRBTEST.COM (short username kudu)
I20260806 09:31:42.797814 6750 webserver.cc:460] Webserver started at http://127.20.113.2:39687/ using document root <none> and password file <none>
I20260806 09:31:42.798564 6750 fs_manager.cc:359] Metadata directory not provided
I20260806 09:31:42.798718 6750 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:31:42.799060 6750 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 09:31:42 dist-test-slave-45dx krb5kdc[5780](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008702, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.2@KRBTEST.COM for HTTP/127.20.113.20@KRBTEST.COM
I20260806 09:31:42.815806 6750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ts-1/data/instance:
uuid: "8a2d3a09b6b0405a850828ac88d2e5e2"
format_stamp: "Formatted at 2026-08-06 09:31:42 on dist-test-slave-45dx"
server_key: "3f8828f9f54821a8fa5b944f6b9dbf54"
server_key_iv: "7977b74e61d0ed0dba10f0e6b217f653"
server_key_version: "kuduclusterkey@0"
I20260806 09:31:42.816882 6750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ts-1/wal/instance:
uuid: "8a2d3a09b6b0405a850828ac88d2e5e2"
format_stamp: "Formatted at 2026-08-06 09:31:42 on dist-test-slave-45dx"
server_key: "3f8828f9f54821a8fa5b944f6b9dbf54"
server_key_iv: "7977b74e61d0ed0dba10f0e6b217f653"
server_key_version: "kuduclusterkey@0"
I20260806 09:31:42.824054 6750 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.000s
I20260806 09:31:42.851750 6784 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:31:42.853587 6750 fs_manager.cc:725] Time spent opening block manager: real 0.012s user 0.007s sys 0.004s
I20260806 09:31:42.853821 6750 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ts-1/wal
uuid: "8a2d3a09b6b0405a850828ac88d2e5e2"
format_stamp: "Formatted at 2026-08-06 09:31:42 on dist-test-slave-45dx"
server_key: "3f8828f9f54821a8fa5b944f6b9dbf54"
server_key_iv: "7977b74e61d0ed0dba10f0e6b217f653"
server_key_version: "kuduclusterkey@0"
I20260806 09:31:42.854080 6750 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:31:42.876200 6750 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:31:42.880563 6750 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:31:42.880883 6750 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:31:42.883895 6750 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:31:42.885584 6750 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:31:42.885697 6750 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:31:42.885875 6750 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:31:42.885951 6750 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:31:42.927923 6750 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.2:44723
I20260806 09:31:42.927943 6937 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.2:44723 every 8 connection(s)
I20260806 09:31:42.929469 6750 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ts-1/data/info.pb
I20260806 09:31:42.935101 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 6750
I20260806 09:31:42.935365 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ts-1/wal/instance
I20260806 09:31:42.936148 20932 external_mini_cluster.cc:1428] Setting key 15a202d3df620b82d071be6541b7957e
Aug 06 09:31:42 dist-test-slave-45dx krb5kdc[5780](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008702, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.2@KRBTEST.COM for kudu/127.20.113.62@KRBTEST.COM
I20260806 09:31:42.974459 6943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:31:42.933175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.2:38557 (local address 127.20.113.62:40899)
0806 09:31:42.957691 (+ 24516us) server_negotiation.cc:206] Beginning negotiation
0806 09:31:42.957697 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:31:42.957718 (+ 21us) server_negotiation.cc:407] Connection header received
0806 09:31:42.957784 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:31:42.957790 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:31:42.957886 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:31:42.958003 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:31:42.960276 (+ 2273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:31:42.961492 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:31:42.963375 (+ 1883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:31:42.963768 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:31:42.963928 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:31:42.967487 (+ 3559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:31:42.967492 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:31:42.967496 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:31:42.967530 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:31:42.970293 (+ 2763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:31:42.971719 (+ 1426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:31:42.971725 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:31:42.971728 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:31:42.971872 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:31:42.972610 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:31:42.972617 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:31:42.972620 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:31:42.972925 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:31:42.973095 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0806 09:31:42.973826 (+ 731us) server_negotiation.cc:299] Negotiation successful
0806 09:31:42.974066 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24363,"thread_start_us":24932,"threads_started":1}
I20260806 09:31:42.975610 6946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:31:42.934141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:40899 (local address 127.20.113.2:38557)
0806 09:31:42.943664 (+ 9523us) negotiation.cc:107] Waiting for socket to connect
0806 09:31:42.943722 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0806 09:31:42.957145 (+ 13423us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:31:42.958344 (+ 1199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:31:42.958360 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:31:42.959281 (+ 921us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:31:42.960051 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:31:42.960071 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:31:42.961681 (+ 1610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:31:42.961686 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:31:42.963205 (+ 1519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:31:42.963226 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:31:42.964028 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:31:42.964037 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:31:42.964359 (+ 322us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:31:42.965136 (+ 777us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:31:42.965159 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:31:42.967240 (+ 2081us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:31:42.971089 (+ 3849us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:31:42.971095 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:31:42.971107 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:31:42.971564 (+ 457us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:31:42.972153 (+ 589us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:31:42.972160 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:31:42.972164 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:31:42.972486 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:31:42.973062 (+ 576us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:31:42.973070 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:31:42.973418 (+ 348us) client_negotiation.cc:769] Sending connection context
0806 09:31:42.973706 (+ 288us) client_negotiation.cc:240] Negotiation successful
0806 09:31:42.974288 (+ 582us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9151,"thread_start_us":12861,"threads_started":1}
I20260806 09:31:42.977344 6938 heartbeater.cc:347] Connected to a master server at 127.20.113.62:40899
I20260806 09:31:42.977701 6938 heartbeater.cc:464] Registering TS with master...
I20260806 09:31:42.978446 6938 heartbeater.cc:511] Master 127.20.113.62:40899 requested a full tablet report, sending...
I20260806 09:31:42.979991 6434 ts_manager.cc:194] Registered new tserver with Master: 8a2d3a09b6b0405a850828ac88d2e5e2 (127.20.113.2:44723)
I20260806 09:31:42.981207 6434 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.113.2@KRBTEST.COM'} at 127.20.113.2:38557
WARNING: no policy specified for kudu/127.20.113.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.113.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.113.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.3.keytab.
Entry for principal kudu/127.20.113.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.3.keytab.
I20260806 09:31:43.036214 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.113.20:58875/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.113.3:0
--local_ip_for_outbound_sockets=127.20.113.3
--webserver_interface=127.20.113.3
--webserver_port=0
--tserver_master_addrs=127.20.113.62:40899
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.3.keytab
--principal=kudu/127.20.113.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:33683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 09:31:43.124840 6739 heartbeater.cc:503] Master 127.20.113.62:40899 was elected leader, sending a full tablet report...
W20260806 09:31:43.403157 6950 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:31:43.403474 6950 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:31:43.403643 6950 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:31:43.410833 6950 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:31:43.410980 6950 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:31:43.411156 6950 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.113.3
I20260806 09:31:43.420989 6950 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:33683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.20.113.20:58875/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.3.keytab
--principal=kudu/127.20.113.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.113.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.113.3
--webserver_port=0
--tserver_master_addrs=127.20.113.62:40899
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.113.3
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:31:43.422415 6950 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:31:43.423806 6950 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:31:43.437798 6961 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:31:43.437879 6958 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:31:43.437893 6956 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:31:43.441740 6950 server_base.cc:1034] running on GCE node
I20260806 09:31:43.442405 6950 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:31:43.443369 6950 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:31:43.444566 6950 hybrid_clock.cc:648] HybridClock initialized: now 1786008703444505 us; error 50 us; skew 500 ppm
I20260806 09:31:43.444895 6950 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:31:43 dist-test-slave-45dx krb5kdc[5780](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008703, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:31:43.448206 6950 init.cc:377] Logged in from keytab as kudu/127.20.113.3@KRBTEST.COM (short username kudu)
I20260806 09:31:43.449741 6950 webserver.cc:460] Webserver started at http://127.20.113.3:32809/ using document root <none> and password file <none>
I20260806 09:31:43.450421 6950 fs_manager.cc:359] Metadata directory not provided
I20260806 09:31:43.450521 6950 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:31:43.450783 6950 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 09:31:43 dist-test-slave-45dx krb5kdc[5780](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008703, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.3@KRBTEST.COM for HTTP/127.20.113.20@KRBTEST.COM
I20260806 09:31:43.466252 6950 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ts-2/data/instance:
uuid: "375a829a81c14acc8fb8f0c67661d7f7"
format_stamp: "Formatted at 2026-08-06 09:31:43 on dist-test-slave-45dx"
server_key: "7950794d78e010b0d7f1a11fc6989662"
server_key_iv: "eaffeacc4184db2e9a94c382e027ce69"
server_key_version: "kuduclusterkey@0"
I20260806 09:31:43.467063 6950 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ts-2/wal/instance:
uuid: "375a829a81c14acc8fb8f0c67661d7f7"
format_stamp: "Formatted at 2026-08-06 09:31:43 on dist-test-slave-45dx"
server_key: "7950794d78e010b0d7f1a11fc6989662"
server_key_iv: "eaffeacc4184db2e9a94c382e027ce69"
server_key_version: "kuduclusterkey@0"
I20260806 09:31:43.472507 6950 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 09:31:43.489737 6969 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:31:43.491506 6950 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260806 09:31:43.491703 6950 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ts-2/wal
uuid: "375a829a81c14acc8fb8f0c67661d7f7"
format_stamp: "Formatted at 2026-08-06 09:31:43 on dist-test-slave-45dx"
server_key: "7950794d78e010b0d7f1a11fc6989662"
server_key_iv: "eaffeacc4184db2e9a94c382e027ce69"
server_key_version: "kuduclusterkey@0"
I20260806 09:31:43.491966 6950 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:31:43.509268 6950 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:31:43.512738 6950 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:31:43.513019 6950 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:31:43.513870 6950 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:31:43.515183 6950 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:31:43.515256 6950 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:31:43.515441 6950 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:31:43.515492 6950 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:31:43.550990 6950 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.3:34637
I20260806 09:31:43.551029 7139 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.3:34637 every 8 connection(s)
I20260806 09:31:43.552277 6950 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ts-2/data/info.pb
I20260806 09:31:43.554937 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 6950
I20260806 09:31:43.555150 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ts-2/wal/instance
I20260806 09:31:43.555802 20932 external_mini_cluster.cc:1428] Setting key 537a536752ca3a9afddb8b35ecb2bc48
Aug 06 09:31:43 dist-test-slave-45dx krb5kdc[5780](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008703, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.3@KRBTEST.COM for kudu/127.20.113.62@KRBTEST.COM
I20260806 09:31:43.573832 7142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:31:43.555018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.3:45359 (local address 127.20.113.62:40899)
0806 09:31:43.555570 (+ 552us) server_negotiation.cc:206] Beginning negotiation
0806 09:31:43.555577 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:31:43.556949 (+ 1372us) server_negotiation.cc:407] Connection header received
0806 09:31:43.559148 (+ 2199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:31:43.559155 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:31:43.559291 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:31:43.559380 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:31:43.560911 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:31:43.561829 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:31:43.563516 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:31:43.563951 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:31:43.564046 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:31:43.567261 (+ 3215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:31:43.567266 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:31:43.567270 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:31:43.567309 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:31:43.570571 (+ 3262us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:31:43.571447 (+ 876us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:31:43.571451 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:31:43.571453 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:31:43.571638 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:31:43.572222 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:31:43.572227 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:31:43.572230 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:31:43.572636 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:31:43.572765 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0806 09:31:43.573405 (+ 640us) server_negotiation.cc:299] Negotiation successful
0806 09:31:43.573567 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":406,"thread_start_us":276,"threads_started":1}
I20260806 09:31:43.574734 7143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:31:43.555787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:40899 (local address 127.20.113.3:45359)
0806 09:31:43.556766 (+ 979us) negotiation.cc:107] Waiting for socket to connect
0806 09:31:43.556823 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0806 09:31:43.558813 (+ 1990us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:31:43.559599 (+ 786us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:31:43.559606 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:31:43.560024 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:31:43.560666 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:31:43.560684 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:31:43.562019 (+ 1335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:31:43.562024 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:31:43.563369 (+ 1345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:31:43.563392 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:31:43.564130 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:31:43.564134 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:31:43.564370 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:31:43.564986 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:31:43.565008 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:31:43.567066 (+ 2058us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:31:43.570721 (+ 3655us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:31:43.570732 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:31:43.570748 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:31:43.571320 (+ 572us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:31:43.571767 (+ 447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:31:43.571773 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:31:43.571776 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:31:43.572082 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:31:43.572769 (+ 687us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:31:43.572775 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:31:43.573029 (+ 254us) client_negotiation.cc:769] Sending connection context
0806 09:31:43.573259 (+ 230us) client_negotiation.cc:240] Negotiation successful
0806 09:31:43.573670 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":626,"thread_start_us":392,"threads_started":1}
I20260806 09:31:43.575977 7140 heartbeater.cc:347] Connected to a master server at 127.20.113.62:40899
I20260806 09:31:43.576272 7140 heartbeater.cc:464] Registering TS with master...
I20260806 09:31:43.576987 7140 heartbeater.cc:511] Master 127.20.113.62:40899 requested a full tablet report, sending...
I20260806 09:31:43.578313 6434 ts_manager.cc:194] Registered new tserver with Master: 375a829a81c14acc8fb8f0c67661d7f7 (127.20.113.3:34637)
I20260806 09:31:43.579144 6434 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.113.3@KRBTEST.COM'} at 127.20.113.3:45359
I20260806 09:31:43.583922 20932 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 09:31:43.585598 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 6552
Aug 06 09:31:43 dist-test-slave-45dx krb5kdc[5780](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008699, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.113.1@KRBTEST.COM
I20260806 09:31:43.601898 7147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:31:43.586433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49984 (local address 127.20.113.1:33547)
0806 09:31:43.587037 (+ 604us) server_negotiation.cc:206] Beginning negotiation
0806 09:31:43.587043 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:31:43.587068 (+ 25us) server_negotiation.cc:407] Connection header received
0806 09:31:43.587179 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:31:43.587185 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:31:43.587354 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:31:43.587476 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:31:43.588961 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:31:43.589950 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:31:43.591464 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:31:43.591979 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:31:43.592090 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:31:43.596119 (+ 4029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:31:43.596130 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:31:43.596142 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:31:43.596172 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:31:43.599258 (+ 3086us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:31:43.599937 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:31:43.599943 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:31:43.599946 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:31:43.600081 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:31:43.600479 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:31:43.600483 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:31:43.600485 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:31:43.600860 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:31:43.601115 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0806 09:31:43.601305 (+ 190us) server_negotiation.cc:299] Negotiation successful
0806 09:31:43.601495 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":467,"thread_start_us":295,"threads_started":1}
I20260806 09:31:43.603044 6734 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:31:43.686694 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 6552
I20260806 09:31:43.709861 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 6750
Aug 06 09:31:43 dist-test-slave-45dx krb5kdc[5780](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008699, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.113.2@KRBTEST.COM
I20260806 09:31:43.731074 7150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:31:43.710823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56882 (local address 127.20.113.2:44723)
0806 09:31:43.711428 (+ 605us) server_negotiation.cc:206] Beginning negotiation
0806 09:31:43.711435 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:31:43.711461 (+ 26us) server_negotiation.cc:407] Connection header received
0806 09:31:43.711523 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:31:43.711530 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:31:43.711748 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:31:43.711880 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:31:43.713293 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:31:43.714310 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:31:43.715693 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:31:43.716229 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:31:43.716389 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:31:43.725758 (+ 9369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:31:43.725771 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:31:43.725786 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:31:43.725820 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:31:43.728565 (+ 2745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:31:43.729218 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:31:43.729225 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:31:43.729227 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:31:43.729352 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:31:43.729676 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:31:43.729679 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:31:43.729681 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:31:43.730016 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:31:43.730324 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0806 09:31:43.730563 (+ 239us) server_negotiation.cc:299] Negotiation successful
0806 09:31:43.730764 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":451,"thread_start_us":317,"threads_started":1}
I20260806 09:31:43.732437 6932 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:31:43.814167 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 6750
I20260806 09:31:43.839959 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 6950
Aug 06 09:31:43 dist-test-slave-45dx krb5kdc[5780](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008699, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.113.3@KRBTEST.COM
I20260806 09:31:43.855880 7152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:31:43.841079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44082 (local address 127.20.113.3:34637)
0806 09:31:43.841699 (+ 620us) server_negotiation.cc:206] Beginning negotiation
0806 09:31:43.841706 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:31:43.841734 (+ 28us) server_negotiation.cc:407] Connection header received
0806 09:31:43.841823 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:31:43.841829 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:31:43.842039 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:31:43.842163 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:31:43.843439 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:31:43.844511 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:31:43.845655 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:31:43.846218 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:31:43.846312 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:31:43.850833 (+ 4521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:31:43.850845 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:31:43.850859 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:31:43.850896 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:31:43.853578 (+ 2682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:31:43.854173 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:31:43.854179 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:31:43.854181 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:31:43.854301 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:31:43.854614 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:31:43.854617 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:31:43.854618 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:31:43.854940 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:31:43.855187 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0806 09:31:43.855384 (+ 197us) server_negotiation.cc:299] Negotiation successful
0806 09:31:43.855586 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":474,"thread_start_us":315,"threads_started":1}
I20260806 09:31:43.857321 7132 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:31:43.940176 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 6950
I20260806 09:31:43.965742 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 6387
I20260806 09:31:43.967023 6477 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:31:44.074000 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 6387
I20260806 09:31:44.105680 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.113.62:40899
--webserver_interface=127.20.113.62
--webserver_port=39251
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab
--principal=kudu/127.20.113.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:33683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.113.62:40899
--ranger_config_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.113.20:58875/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.20.113.20:58875/kms with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:31:44.454597 7155 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:31:44.454885 7155 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:31:44.454993 7155 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:31:44.462731 7155 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 09:31:44.462899 7155 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:31:44.463002 7155 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:31:44.463114 7155 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 09:31:44.463199 7155 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 09:31:44.473196 7155 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:33683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.20.113.20:58875/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.20.113.62:40899
--ranger_config_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab
--principal=kudu/127.20.113.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.113.62:40899
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.113.62
--webserver_port=39251
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:31:44.474571 7155 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:31:44.475966 7155 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:31:44.488207 7165 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:31:44.488343 7169 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:31:44.488238 7164 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:31:44.489415 7155 server_base.cc:1034] running on GCE node
I20260806 09:31:44.490319 7155 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:31:44.491767 7155 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:31:44.493103 7155 hybrid_clock.cc:648] HybridClock initialized: now 1786008704493051 us; error 32 us; skew 500 ppm
I20260806 09:31:44.493477 7155 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:31:44 dist-test-slave-45dx krb5kdc[5780](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008704, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:31:44.496788 7155 init.cc:377] Logged in from keytab as kudu/127.20.113.62@KRBTEST.COM (short username kudu)
I20260806 09:31:44.498317 7155 webserver.cc:460] Webserver started at http://127.20.113.62:39251/ using document root <none> and password file <none>
I20260806 09:31:44.498984 7155 fs_manager.cc:359] Metadata directory not provided
I20260806 09:31:44.499131 7155 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 09:31:44 dist-test-slave-45dx krb5kdc[5780](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008704, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.62@KRBTEST.COM for HTTP/127.20.113.20@KRBTEST.COM
I20260806 09:31:44.549595 7155 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 09:31:44.552711 7182 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:31:44.554203 7155 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260806 09:31:44.554407 7155 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/master-0/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/master-0/wal
uuid: "4acecc18510e46919fbd36bbdbe95ed7"
format_stamp: "Formatted at 2026-08-06 09:31:40 on dist-test-slave-45dx"
server_key: "ec777597967426e54ddc942f6624c026"
server_key_iv: "b2ae7a4c207a74654b7cb4ff44f12c05"
server_key_version: "kuduclusterkey@0"
I20260806 09:31:44.555369 7155 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:31:44.579072 7155 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:31:44.583361 7155 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:31:44.583671 7155 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:31:44.603482 7155 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.62:40899
I20260806 09:31:44.603489 7255 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.62:40899 every 8 connection(s)
I20260806 09:31:44.605142 7155 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
I20260806 09:31:44.610944 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 7155
I20260806 09:31:44.612087 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.113.20:58875/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.113.1:33547
--local_ip_for_outbound_sockets=127.20.113.1
--tserver_master_addrs=127.20.113.62:40899
--webserver_port=46393
--webserver_interface=127.20.113.1
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.1.keytab
--principal=kudu/127.20.113.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:33683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.20.113.20:58875/kms with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 09:31:44.612901 7257 sys_catalog.cc:263] Verifying existing consensus state
I20260806 09:31:44.618232 7257 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4acecc18510e46919fbd36bbdbe95ed7: Bootstrap starting.
I20260806 09:31:44.639729 7257 log.cc:824] T 00000000000000000000000000000000 P 4acecc18510e46919fbd36bbdbe95ed7: Log is configured to *not* fsync() on all Append() calls
I20260806 09:31:44.648885 7257 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4acecc18510e46919fbd36bbdbe95ed7: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 09:31:44.649449 7257 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4acecc18510e46919fbd36bbdbe95ed7: Bootstrap complete.
I20260806 09:31:44.658808 7257 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4acecc18510e46919fbd36bbdbe95ed7 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4acecc18510e46919fbd36bbdbe95ed7" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 40899 } }
I20260806 09:31:44.659747 7257 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4acecc18510e46919fbd36bbdbe95ed7 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4acecc18510e46919fbd36bbdbe95ed7, State: Initialized, Role: FOLLOWER
I20260806 09:31:44.660468 7257 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4acecc18510e46919fbd36bbdbe95ed7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4acecc18510e46919fbd36bbdbe95ed7" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 40899 } }
I20260806 09:31:44.660696 7257 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4acecc18510e46919fbd36bbdbe95ed7 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 09:31:44.660842 7257 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4acecc18510e46919fbd36bbdbe95ed7 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 09:31:44.661032 7257 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4acecc18510e46919fbd36bbdbe95ed7 [term 1 FOLLOWER]: Advancing to term 2
I20260806 09:31:44.664978 7257 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4acecc18510e46919fbd36bbdbe95ed7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4acecc18510e46919fbd36bbdbe95ed7" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 40899 } }
I20260806 09:31:44.665465 7257 leader_election.cc:302] T 00000000000000000000000000000000 P 4acecc18510e46919fbd36bbdbe95ed7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4acecc18510e46919fbd36bbdbe95ed7; no voters:
I20260806 09:31:44.666126 7257 leader_election.cc:288] T 00000000000000000000000000000000 P 4acecc18510e46919fbd36bbdbe95ed7 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 09:31:44.666261 7262 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4acecc18510e46919fbd36bbdbe95ed7 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 09:31:44.667603 7262 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4acecc18510e46919fbd36bbdbe95ed7 [term 2 LEADER]: Becoming Leader. State: Replica: 4acecc18510e46919fbd36bbdbe95ed7, State: Running, Role: LEADER
I20260806 09:31:44.668437 7257 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4acecc18510e46919fbd36bbdbe95ed7 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 09:31:44.668263 7262 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4acecc18510e46919fbd36bbdbe95ed7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4acecc18510e46919fbd36bbdbe95ed7" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 40899 } }
I20260806 09:31:44.671062 7263 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4acecc18510e46919fbd36bbdbe95ed7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "4acecc18510e46919fbd36bbdbe95ed7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4acecc18510e46919fbd36bbdbe95ed7" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 40899 } } }
I20260806 09:31:44.671537 7263 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4acecc18510e46919fbd36bbdbe95ed7 [sys.catalog]: This master's current role is: LEADER
I20260806 09:31:44.671489 7264 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4acecc18510e46919fbd36bbdbe95ed7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4acecc18510e46919fbd36bbdbe95ed7. Latest consensus state: current_term: 2 leader_uuid: "4acecc18510e46919fbd36bbdbe95ed7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4acecc18510e46919fbd36bbdbe95ed7" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 40899 } } }
I20260806 09:31:44.671864 7264 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4acecc18510e46919fbd36bbdbe95ed7 [sys.catalog]: This master's current role is: LEADER
I20260806 09:31:44.674500 7269 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 09:31:44.682796 7269 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 09:31:44.683998 7257 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 09:31:44.684378 7269 catalog_manager.cc:1259] Loaded cluster ID: 9f7cf3db701d43c5b764a7d82e4a07b8
I20260806 09:31:44.684473 7269 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 09:31:44.698055 7269 catalog_manager.cc:1504] Loading token signing keys...
I20260806 09:31:44.702909 7269 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 4acecc18510e46919fbd36bbdbe95ed7: Loaded TSK: 0
I20260806 09:31:44.704733 7269 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 09:31:45.079535 7259 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:31:45.079919 7259 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:31:45.080086 7259 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:31:45.091588 7259 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:31:45.091733 7259 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:31:45.091930 7259 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.113.1
I20260806 09:31:45.107728 7259 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:33683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.20.113.20:58875/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.1.keytab
--principal=kudu/127.20.113.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.113.1:33547
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.113.1
--webserver_port=46393
--tserver_master_addrs=127.20.113.62:40899
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.113.1
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:31:45.109550 7259 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:31:45.111400 7259 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:31:45.145298 7312 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:31:45.149047 7310 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:31:45.159521 7315 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:31:45.161137 7259 server_base.cc:1034] running on GCE node
I20260806 09:31:45.162187 7259 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:31:45.172575 7259 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:31:45.174150 7259 hybrid_clock.cc:648] HybridClock initialized: now 1786008705174062 us; error 78 us; skew 500 ppm
I20260806 09:31:45.174638 7259 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:31:45 dist-test-slave-45dx krb5kdc[5780](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008705, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:31:45.179344 7259 init.cc:377] Logged in from keytab as kudu/127.20.113.1@KRBTEST.COM (short username kudu)
I20260806 09:31:45.202335 7259 webserver.cc:460] Webserver started at http://127.20.113.1:46393/ using document root <none> and password file <none>
I20260806 09:31:45.203636 7259 fs_manager.cc:359] Metadata directory not provided
I20260806 09:31:45.203927 7259 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 09:31:45 dist-test-slave-45dx krb5kdc[5780](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008705, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.1@KRBTEST.COM for HTTP/127.20.113.20@KRBTEST.COM
I20260806 09:31:45.247687 7259 fs_manager.cc:709] Time spent opening directory manager: real 0.015s user 0.004s sys 0.008s
I20260806 09:31:45.251060 7330 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:31:45.254496 7259 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.000s
I20260806 09:31:45.254711 7259 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ts-0/wal
uuid: "66040518f530435183215ce1112bd084"
format_stamp: "Formatted at 2026-08-06 09:31:41 on dist-test-slave-45dx"
server_key: "3c7b5a235d1f9520a4fb2e7b22c64352"
server_key_iv: "16a826bc6d0365c0ab789d79a3b53cbc"
server_key_version: "kuduclusterkey@0"
I20260806 09:31:45.256361 7259 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:31:45.312948 7259 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:31:45.318441 7259 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:31:45.318759 7259 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:31:45.321717 7259 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:31:45.324836 7259 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:31:45.324954 7259 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:31:45.325224 7259 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:31:45.325309 7259 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:31:45.380367 7259 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.1:33547
I20260806 09:31:45.380502 7490 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.1:33547 every 8 connection(s)
I20260806 09:31:45.381948 7259 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ts-0/data/info.pb
I20260806 09:31:45.388514 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 7259
I20260806 09:31:45.389494 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.113.20:58875/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.113.2:44723
--local_ip_for_outbound_sockets=127.20.113.2
--tserver_master_addrs=127.20.113.62:40899
--webserver_port=39687
--webserver_interface=127.20.113.2
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.2.keytab
--principal=kudu/127.20.113.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:33683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.20.113.20:58875/kms with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 09:31:45 dist-test-slave-45dx krb5kdc[5780](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008705, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.1@KRBTEST.COM for kudu/127.20.113.62@KRBTEST.COM
I20260806 09:31:45.438385 7496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:31:45.388917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.1:59169 (local address 127.20.113.62:40899)
0806 09:31:45.406808 (+ 17891us) server_negotiation.cc:206] Beginning negotiation
0806 09:31:45.406823 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 09:31:45.406874 (+ 51us) server_negotiation.cc:407] Connection header received
0806 09:31:45.408513 (+ 1639us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:31:45.408527 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:31:45.408906 (+ 379us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:31:45.409299 (+ 393us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:31:45.411267 (+ 1968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:31:45.413388 (+ 2121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:31:45.416130 (+ 2742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:31:45.417386 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:31:45.417587 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:31:45.429975 (+ 12388us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:31:45.429985 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:31:45.430004 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:31:45.430050 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:31:45.433289 (+ 3239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:31:45.434149 (+ 860us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:31:45.434158 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:31:45.434164 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:31:45.434329 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:31:45.434845 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:31:45.434851 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:31:45.434854 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:31:45.435550 (+ 696us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:31:45.435747 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0806 09:31:45.436952 (+ 1205us) server_negotiation.cc:299] Negotiation successful
0806 09:31:45.437300 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17597,"thread_start_us":17387,"threads_started":1}
I20260806 09:31:45.440269 7495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:31:45.387127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:40899 (local address 127.20.113.1:59169)
0806 09:31:45.405044 (+ 17917us) negotiation.cc:107] Waiting for socket to connect
0806 09:31:45.405112 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0806 09:31:45.407738 (+ 2626us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:31:45.409475 (+ 1737us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:31:45.409487 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:31:45.410190 (+ 703us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:31:45.411013 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:31:45.411041 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:31:45.413595 (+ 2554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:31:45.413604 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:31:45.415889 (+ 2285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:31:45.415909 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:31:45.417997 (+ 2088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:31:45.418007 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:31:45.418319 (+ 312us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:31:45.424620 (+ 6301us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:31:45.424660 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:31:45.429576 (+ 4916us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:31:45.433486 (+ 3910us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:31:45.433496 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:31:45.433515 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:31:45.433973 (+ 458us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:31:45.434465 (+ 492us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:31:45.434470 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:31:45.434474 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:31:45.434702 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:31:45.435854 (+ 1152us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:31:45.435862 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:31:45.436331 (+ 469us) client_negotiation.cc:769] Sending connection context
0806 09:31:45.438866 (+ 2535us) client_negotiation.cc:240] Negotiation successful
0806 09:31:45.439322 (+ 456us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17520,"thread_start_us":17268,"threads_started":1}
I20260806 09:31:45.443773 7491 heartbeater.cc:347] Connected to a master server at 127.20.113.62:40899
I20260806 09:31:45.444108 7491 heartbeater.cc:464] Registering TS with master...
I20260806 09:31:45.444960 7491 heartbeater.cc:511] Master 127.20.113.62:40899 requested a full tablet report, sending...
W20260806 09:31:45.448007 7491 heartbeater.cc:644] Failed to heartbeat to 127.20.113.62:40899 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 09:31:45.448275 7491 heartbeater.cc:464] Registering TS with master...
I20260806 09:31:45.448624 7491 heartbeater.cc:511] Master 127.20.113.62:40899 requested a full tablet report, sending...
I20260806 09:31:45.449894 7491 heartbeater.cc:464] Registering TS with master...
I20260806 09:31:45.450179 7491 heartbeater.cc:511] Master 127.20.113.62:40899 requested a full tablet report, sending...
W20260806 09:31:45.451234 7491 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 09:31:45.855381 7497 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:31:45.855688 7497 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:31:45.855865 7497 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:31:45.866206 7497 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:31:45.866341 7497 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:31:45.866539 7497 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.113.2
I20260806 09:31:45.881436 7497 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:33683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.20.113.20:58875/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.2.keytab
--principal=kudu/127.20.113.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.113.2:44723
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.113.2
--webserver_port=39687
--tserver_master_addrs=127.20.113.62:40899
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.113.2
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:31:45.883189 7497 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:31:45.884953 7497 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:31:45.906778 7522 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:31:45.906723 7524 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:31:45.908499 7530 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:31:45.909214 7497 server_base.cc:1034] running on GCE node
I20260806 09:31:45.909946 7497 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:31:45.911031 7497 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:31:45.912375 7497 hybrid_clock.cc:648] HybridClock initialized: now 1786008705912298 us; error 48 us; skew 500 ppm
I20260806 09:31:45.912707 7497 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:31:45 dist-test-slave-45dx krb5kdc[5780](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008705, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:31:45.915954 7497 init.cc:377] Logged in from keytab as kudu/127.20.113.2@KRBTEST.COM (short username kudu)
I20260806 09:31:45.917811 7497 webserver.cc:460] Webserver started at http://127.20.113.2:39687/ using document root <none> and password file <none>
I20260806 09:31:45.918529 7497 fs_manager.cc:359] Metadata directory not provided
I20260806 09:31:45.918694 7497 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 09:31:45 dist-test-slave-45dx krb5kdc[5780](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008705, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.2@KRBTEST.COM for HTTP/127.20.113.20@KRBTEST.COM
I20260806 09:31:45.954634 7497 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.002s sys 0.001s
I20260806 09:31:45.975256 7541 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:31:45.977357 7497 fs_manager.cc:725] Time spent opening block manager: real 0.021s user 0.010s sys 0.008s
I20260806 09:31:45.977613 7497 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ts-1/wal
uuid: "8a2d3a09b6b0405a850828ac88d2e5e2"
format_stamp: "Formatted at 2026-08-06 09:31:42 on dist-test-slave-45dx"
server_key: "3f8828f9f54821a8fa5b944f6b9dbf54"
server_key_iv: "7977b74e61d0ed0dba10f0e6b217f653"
server_key_version: "kuduclusterkey@0"
I20260806 09:31:45.979197 7497 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
Aug 06 09:31:45 dist-test-slave-45dx krb5kdc[5780](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786008705, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:31:46.016160 7497 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:31:46.021443 7497 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:31:46.021781 7497 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:31:46.029313 7497 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:31:46.037290 7497 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:31:46.037417 7497 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:31:46.037544 7497 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:31:46.037624 7497 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:31:46.124156 7497 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.2:44723
I20260806 09:31:46.124209 7713 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.2:44723 every 8 connection(s)
I20260806 09:31:46.126255 7497 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ts-1/data/info.pb
I20260806 09:31:46.127832 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 7497
I20260806 09:31:46.128808 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.113.20:58875/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.113.3:34637
--local_ip_for_outbound_sockets=127.20.113.3
--tserver_master_addrs=127.20.113.62:40899
--webserver_port=32809
--webserver_interface=127.20.113.3
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.3.keytab
--principal=kudu/127.20.113.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:33683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.20.113.20:58875/kms with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 09:31:46 dist-test-slave-45dx krb5kdc[5780](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008705, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.2@KRBTEST.COM for kudu/127.20.113.62@KRBTEST.COM
I20260806 09:31:46.175199 7716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:31:46.136621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.2:33255 (local address 127.20.113.62:40899)
0806 09:31:46.138684 (+ 2063us) server_negotiation.cc:206] Beginning negotiation
0806 09:31:46.138691 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:31:46.145113 (+ 6422us) server_negotiation.cc:407] Connection header received
0806 09:31:46.151506 (+ 6393us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:31:46.151511 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:31:46.151589 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:31:46.151738 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:31:46.153834 (+ 2096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:31:46.155096 (+ 1262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:31:46.158142 (+ 3046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:31:46.159007 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:31:46.159154 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:31:46.165068 (+ 5914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:31:46.165077 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:31:46.165085 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:31:46.165136 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:31:46.169627 (+ 4491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:31:46.170728 (+ 1101us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:31:46.170739 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:31:46.170745 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:31:46.170937 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:31:46.171608 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:31:46.171617 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:31:46.171623 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:31:46.172076 (+ 453us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:31:46.172254 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0806 09:31:46.174217 (+ 1963us) server_negotiation.cc:299] Negotiation successful
0806 09:31:46.174519 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1908,"thread_start_us":1776,"threads_started":1}
I20260806 09:31:46.178251 7717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:31:46.136239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:40899 (local address 127.20.113.2:33255)
0806 09:31:46.144917 (+ 8678us) negotiation.cc:107] Waiting for socket to connect
0806 09:31:46.144968 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 09:31:46.146814 (+ 1846us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:31:46.152036 (+ 5222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:31:46.152046 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:31:46.152623 (+ 577us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:31:46.153565 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:31:46.153596 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:31:46.155337 (+ 1741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:31:46.155345 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:31:46.157867 (+ 2522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:31:46.157890 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:31:46.159507 (+ 1617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:31:46.159519 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:31:46.159926 (+ 407us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:31:46.160971 (+ 1045us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:31:46.161009 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:31:46.164783 (+ 3774us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:31:46.169800 (+ 5017us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:31:46.169814 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:31:46.169837 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:31:46.170553 (+ 716us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:31:46.171089 (+ 536us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:31:46.171097 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:31:46.171104 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:31:46.171442 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:31:46.172404 (+ 962us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:31:46.172416 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:31:46.172945 (+ 529us) client_negotiation.cc:769] Sending connection context
0806 09:31:46.173375 (+ 430us) client_negotiation.cc:240] Negotiation successful
0806 09:31:46.176428 (+ 3053us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8374,"thread_start_us":7608,"threads_started":1}
I20260806 09:31:46.180786 7714 heartbeater.cc:347] Connected to a master server at 127.20.113.62:40899
I20260806 09:31:46.181177 7714 heartbeater.cc:464] Registering TS with master...
I20260806 09:31:46.182312 7714 heartbeater.cc:511] Master 127.20.113.62:40899 requested a full tablet report, sending...
I20260806 09:31:46.184708 7207 ts_manager.cc:194] Registered new tserver with Master: 8a2d3a09b6b0405a850828ac88d2e5e2 (127.20.113.2:44723)
I20260806 09:31:46.187115 7207 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.113.2@KRBTEST.COM'} at 127.20.113.2:33255
I20260806 09:31:46.451393 7491 heartbeater.cc:464] Registering TS with master...
I20260806 09:31:46.451860 7491 heartbeater.cc:511] Master 127.20.113.62:40899 requested a full tablet report, sending...
I20260806 09:31:46.453104 7207 ts_manager.cc:194] Registered new tserver with Master: 66040518f530435183215ce1112bd084 (127.20.113.1:33547)
I20260806 09:31:46.454304 7207 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.113.1@KRBTEST.COM'} at 127.20.113.1:59169
Aug 06 09:31:46 dist-test-slave-45dx krb5kdc[5780](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008705, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.62@KRBTEST.COM for HTTP/127.20.113.20@KRBTEST.COM
W20260806 09:31:46.605489 7718 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:31:46.605857 7718 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:31:46.606030 7718 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:31:46.617637 7718 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:31:46.617774 7718 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:31:46.617971 7718 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.113.3
I20260806 09:31:46.633642 7718 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:33683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.20.113.20:58875/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.3.keytab
--principal=kudu/127.20.113.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.113.3:34637
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.113.3
--webserver_port=32809
--tserver_master_addrs=127.20.113.62:40899
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.113.3
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:31:46.635342 7718 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:31:46.637221 7718 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:31:46.655238 7726 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:31:46.656181 7724 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:31:46.664788 7718 server_base.cc:1034] running on GCE node
W20260806 09:31:46.665186 7730 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:31:46.665839 7718 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:31:46.666971 7718 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:31:46.668198 7718 hybrid_clock.cc:648] HybridClock initialized: now 1786008706668126 us; error 56 us; skew 500 ppm
I20260806 09:31:46.668563 7718 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:31:46 dist-test-slave-45dx krb5kdc[5780](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008706, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:31:46.672139 7718 init.cc:377] Logged in from keytab as kudu/127.20.113.3@KRBTEST.COM (short username kudu)
I20260806 09:31:46.674154 7718 webserver.cc:460] Webserver started at http://127.20.113.3:32809/ using document root <none> and password file <none>
I20260806 09:31:46.674959 7718 fs_manager.cc:359] Metadata directory not provided
I20260806 09:31:46.675133 7718 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 09:31:46 dist-test-slave-45dx krb5kdc[5780](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008706, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.3@KRBTEST.COM for HTTP/127.20.113.20@KRBTEST.COM
I20260806 09:31:46.720580 7718 fs_manager.cc:709] Time spent opening directory manager: real 0.030s user 0.015s sys 0.012s
I20260806 09:31:46.730746 7744 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:31:46.732427 7718 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.010s sys 0.000s
I20260806 09:31:46.732611 7718 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ts-2/wal
uuid: "375a829a81c14acc8fb8f0c67661d7f7"
format_stamp: "Formatted at 2026-08-06 09:31:43 on dist-test-slave-45dx"
server_key: "7950794d78e010b0d7f1a11fc6989662"
server_key_iv: "eaffeacc4184db2e9a94c382e027ce69"
server_key_version: "kuduclusterkey@0"
I20260806 09:31:46.733700 7718 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:31:46.746592 7718 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:31:46.750676 7718 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:31:46.750983 7718 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:31:46.752044 7718 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:31:46.753551 7718 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:31:46.753674 7718 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:31:46.753806 7718 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:31:46.753913 7718 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:31:46.812793 7718 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.3:34637
I20260806 09:31:46.812842 7909 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.3:34637 every 8 connection(s)
I20260806 09:31:46.814592 7718 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ts-2/data/info.pb
I20260806 09:31:46.816195 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 7718
I20260806 09:31:46.848079 7913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:31:46.817487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57442 (local address 127.20.113.62:40899)
0806 09:31:46.821661 (+ 4174us) server_negotiation.cc:206] Beginning negotiation
0806 09:31:46.821668 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:31:46.821688 (+ 20us) server_negotiation.cc:407] Connection header received
0806 09:31:46.821785 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:31:46.821809 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:31:46.821925 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:31:46.822039 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:31:46.826553 (+ 4514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:31:46.828833 (+ 2280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:31:46.831616 (+ 2783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:31:46.832543 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:31:46.832689 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:31:46.839959 (+ 7270us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:31:46.839966 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:31:46.839971 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:31:46.840027 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:31:46.843741 (+ 3714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:31:46.844866 (+ 1125us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:31:46.844871 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:31:46.844874 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:31:46.845030 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:31:46.845560 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:31:46.845565 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:31:46.845569 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:31:46.845926 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:31:46.846190 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0806 09:31:46.847451 (+ 1261us) server_negotiation.cc:299] Negotiation successful
0806 09:31:46.847683 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4045,"thread_start_us":5185,"threads_started":1}
Aug 06 09:31:46 dist-test-slave-45dx krb5kdc[5780](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008706, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.3@KRBTEST.COM for kudu/127.20.113.62@KRBTEST.COM
I20260806 09:31:46.860299 7915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:31:46.821881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.3:41445 (local address 127.20.113.62:40899)
0806 09:31:46.828495 (+ 6614us) server_negotiation.cc:206] Beginning negotiation
0806 09:31:46.828504 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 09:31:46.835499 (+ 6995us) server_negotiation.cc:407] Connection header received
0806 09:31:46.835542 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:31:46.835546 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:31:46.835623 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:31:46.835746 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:31:46.838164 (+ 2418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:31:46.839539 (+ 1375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:31:46.842113 (+ 2574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:31:46.842864 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:31:46.842968 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:31:46.853143 (+ 10175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:31:46.853147 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:31:46.853150 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:31:46.853185 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:31:46.856517 (+ 3332us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:31:46.857491 (+ 974us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:31:46.857497 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:31:46.857501 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:31:46.857648 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:31:46.858203 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:31:46.858208 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:31:46.858212 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:31:46.858580 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:31:46.858706 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0806 09:31:46.859663 (+ 957us) server_negotiation.cc:299] Negotiation successful
0806 09:31:46.859890 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6463,"spinlock_wait_cycles":5504,"thread_start_us":6446,"threads_started":1}
I20260806 09:31:46.863816 7916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:31:46.822407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:40899 (local address 127.20.113.3:41445)
0806 09:31:46.833441 (+ 11034us) negotiation.cc:107] Waiting for socket to connect
0806 09:31:46.833498 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0806 09:31:46.835228 (+ 1730us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:31:46.836027 (+ 799us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:31:46.836039 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:31:46.836706 (+ 667us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:31:46.837752 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:31:46.837784 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:31:46.839729 (+ 1945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:31:46.839735 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:31:46.841941 (+ 2206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:31:46.841957 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:31:46.843342 (+ 1385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:31:46.843349 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:31:46.843645 (+ 296us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:31:46.845853 (+ 2208us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:31:46.845884 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:31:46.852812 (+ 6928us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:31:46.856690 (+ 3878us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:31:46.856699 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:31:46.856717 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:31:46.857333 (+ 616us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:31:46.857799 (+ 466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:31:46.857804 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:31:46.857808 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:31:46.858070 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:31:46.858796 (+ 726us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:31:46.858804 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:31:46.859218 (+ 414us) client_negotiation.cc:769] Sending connection context
0806 09:31:46.860464 (+ 1246us) client_negotiation.cc:240] Negotiation successful
0806 09:31:46.861061 (+ 597us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10755,"thread_start_us":10535,"threads_started":1}
I20260806 09:31:46.868794 7911 heartbeater.cc:347] Connected to a master server at 127.20.113.62:40899
I20260806 09:31:46.869120 7911 heartbeater.cc:464] Registering TS with master...
I20260806 09:31:46.870448 7911 heartbeater.cc:511] Master 127.20.113.62:40899 requested a full tablet report, sending...
I20260806 09:31:46.872202 7207 ts_manager.cc:194] Registered new tserver with Master: 375a829a81c14acc8fb8f0c67661d7f7 (127.20.113.3:34637)
I20260806 09:31:46.873350 7207 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.113.3@KRBTEST.COM'} at 127.20.113.3:41445
Aug 06 09:31:46 dist-test-slave-45dx krb5kdc[5780](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008705, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.62@KRBTEST.COM for HTTP/127.20.113.20@KRBTEST.COM
I20260806 09:31:46.876119 20932 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 09:31:46.947665 7915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:31:46.908727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57454 (local address 127.20.113.62:40899)
0806 09:31:46.908945 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0806 09:31:46.908952 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:31:46.925007 (+ 16055us) server_negotiation.cc:407] Connection header received
0806 09:31:46.925254 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:31:46.925260 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:31:46.925349 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:31:46.925485 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:31:46.927310 (+ 1825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:31:46.928520 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:31:46.930644 (+ 2124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:31:46.931281 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:31:46.931418 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:31:46.935164 (+ 3746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:31:46.935171 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:31:46.935176 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:31:46.935221 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:31:46.941513 (+ 6292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:31:46.942404 (+ 891us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:31:46.942410 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:31:46.942414 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:31:46.942576 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:31:46.945152 (+ 2576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:31:46.945158 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:31:46.945161 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:31:46.945604 (+ 443us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:31:46.945849 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0806 09:31:46.947052 (+ 1203us) server_negotiation.cc:299] Negotiation successful
0806 09:31:46.947293 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260806 09:31:46.960215 7207 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57454:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 09:31:46.975651 7207 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 09:31:47.068005 7935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:31:47.009662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.3:34637 (local address 127.0.0.1:44096)
0806 09:31:47.025840 (+ 16178us) negotiation.cc:107] Waiting for socket to connect
0806 09:31:47.025887 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 09:31:47.026106 (+ 219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:31:47.059534 (+ 33428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:31:47.059538 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:31:47.059572 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:31:47.060019 (+ 447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:31:47.060034 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:31:47.062570 (+ 2536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:31:47.062576 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:31:47.064898 (+ 2322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:31:47.064914 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:31:47.066983 (+ 2069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:31:47.066991 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:31:47.067201 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:31:47.067262 (+ 61us) client_negotiation.cc:769] Sending connection context
0806 09:31:47.067377 (+ 115us) client_negotiation.cc:240] Negotiation successful
0806 09:31:47.067570 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15869,"mutex_wait_us":3,"thread_start_us":16633,"threads_started":1}
I20260806 09:31:47.069288 7937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:31:47.009980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44096 (local address 127.20.113.3:34637)
0806 09:31:47.058866 (+ 48886us) server_negotiation.cc:206] Beginning negotiation
0806 09:31:47.058875 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 09:31:47.058910 (+ 35us) server_negotiation.cc:407] Connection header received
0806 09:31:47.059024 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:31:47.059031 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:31:47.059291 (+ 260us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:31:47.060248 (+ 957us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:31:47.060582 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:31:47.062195 (+ 1613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:31:47.065181 (+ 2986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:31:47.066707 (+ 1526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:31:47.068310 (+ 1603us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:31:47.068571 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0806 09:31:47.068648 (+ 77us) server_negotiation.cc:299] Negotiation successful
0806 09:31:47.068882 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48669,"thread_start_us":52281,"threads_started":1}
I20260806 09:31:47.069564 7941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:31:47.033538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.2:44723 (local address 127.0.0.1:56886)
0806 09:31:47.049573 (+ 16035us) negotiation.cc:107] Waiting for socket to connect
0806 09:31:47.049594 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 09:31:47.049739 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:31:47.057320 (+ 7581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:31:47.057325 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:31:47.057355 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:31:47.057728 (+ 373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:31:47.057742 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:31:47.062850 (+ 5108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:31:47.062855 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:31:47.066103 (+ 3248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:31:47.066117 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:31:47.068554 (+ 2437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:31:47.068560 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:31:47.068718 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:31:47.068749 (+ 31us) client_negotiation.cc:769] Sending connection context
0806 09:31:47.068827 (+ 78us) client_negotiation.cc:240] Negotiation successful
0806 09:31:47.068983 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15891,"mutex_wait_us":1094,"thread_start_us":9938,"threads_started":1}
I20260806 09:31:47.073096 7944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:31:47.027486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56886 (local address 127.20.113.2:44723)
0806 09:31:47.044814 (+ 17328us) server_negotiation.cc:206] Beginning negotiation
0806 09:31:47.044823 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 09:31:47.056398 (+ 11575us) server_negotiation.cc:407] Connection header received
0806 09:31:47.056530 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:31:47.056538 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:31:47.056827 (+ 289us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:31:47.057058 (+ 231us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:31:47.057938 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:31:47.062529 (+ 4591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:31:47.066289 (+ 3760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:31:47.067760 (+ 1471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:31:47.067912 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:31:47.072524 (+ 4612us) server_negotiation.cc:1035] Waiting for connection context
0806 09:31:47.072610 (+ 86us) server_negotiation.cc:299] Negotiation successful
0806 09:31:47.072779 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17149,"thread_start_us":17339,"threads_started":1}
I20260806 09:31:47.073477 7816 tablet_service.cc:1467] Processing CreateTablet for tablet a1824279d6d444c5a53ea019c5a22073 (DEFAULT_TABLE table=test-table [id=ea6a40176ab841dbb2b1fd533ca3428c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:31:47.075217 7816 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1824279d6d444c5a53ea019c5a22073. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:31:47.077739 7615 tablet_service.cc:1467] Processing CreateTablet for tablet a1824279d6d444c5a53ea019c5a22073 (DEFAULT_TABLE table=test-table [id=ea6a40176ab841dbb2b1fd533ca3428c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:31:47.079124 7615 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1824279d6d444c5a53ea019c5a22073. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:31:47.086793 7946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:31:47.012118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.1:33547 (local address 127.0.0.1:49990)
0806 09:31:47.043206 (+ 31088us) negotiation.cc:107] Waiting for socket to connect
0806 09:31:47.043230 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 09:31:47.043385 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:31:47.050805 (+ 7420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:31:47.050812 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:31:47.050844 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:31:47.051284 (+ 440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:31:47.051300 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:31:47.058742 (+ 7442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:31:47.058748 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:31:47.061412 (+ 2664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:31:47.061427 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:31:47.085769 (+ 24342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:31:47.085776 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:31:47.085965 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:31:47.086141 (+ 176us) client_negotiation.cc:769] Sending connection context
0806 09:31:47.086245 (+ 104us) client_negotiation.cc:240] Negotiation successful
0806 09:31:47.086422 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30936,"thread_start_us":37715,"threads_started":1}
I20260806 09:31:47.088097 7943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:31:47.012821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49990 (local address 127.20.113.1:33547)
0806 09:31:47.047958 (+ 35137us) server_negotiation.cc:206] Beginning negotiation
0806 09:31:47.047967 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 09:31:47.048002 (+ 35us) server_negotiation.cc:407] Connection header received
0806 09:31:47.048120 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:31:47.048129 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:31:47.050228 (+ 2099us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:31:47.050495 (+ 267us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:31:47.056537 (+ 6042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:31:47.058236 (+ 1699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:31:47.063344 (+ 5108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:31:47.085175 (+ 21831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:31:47.085347 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:31:47.085600 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0806 09:31:47.087483 (+ 1883us) server_negotiation.cc:299] Negotiation successful
0806 09:31:47.087681 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34936,"thread_start_us":35493,"threads_started":1}
I20260806 09:31:47.101310 7395 tablet_service.cc:1467] Processing CreateTablet for tablet a1824279d6d444c5a53ea019c5a22073 (DEFAULT_TABLE table=test-table [id=ea6a40176ab841dbb2b1fd533ca3428c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:31:47.102902 7395 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1824279d6d444c5a53ea019c5a22073. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:31:47.103772 7949 tablet_bootstrap.cc:492] T a1824279d6d444c5a53ea019c5a22073 P 375a829a81c14acc8fb8f0c67661d7f7: Bootstrap starting.
I20260806 09:31:47.112008 7949 tablet_bootstrap.cc:654] T a1824279d6d444c5a53ea019c5a22073 P 375a829a81c14acc8fb8f0c67661d7f7: Neither blocks nor log segments found. Creating new log.
I20260806 09:31:47.121803 7949 log.cc:824] T a1824279d6d444c5a53ea019c5a22073 P 375a829a81c14acc8fb8f0c67661d7f7: Log is configured to *not* fsync() on all Append() calls
I20260806 09:31:47.122809 7951 tablet_bootstrap.cc:492] T a1824279d6d444c5a53ea019c5a22073 P 8a2d3a09b6b0405a850828ac88d2e5e2: Bootstrap starting.
I20260806 09:31:47.128244 7951 tablet_bootstrap.cc:654] T a1824279d6d444c5a53ea019c5a22073 P 8a2d3a09b6b0405a850828ac88d2e5e2: Neither blocks nor log segments found. Creating new log.
I20260806 09:31:47.130414 7951 log.cc:824] T a1824279d6d444c5a53ea019c5a22073 P 8a2d3a09b6b0405a850828ac88d2e5e2: Log is configured to *not* fsync() on all Append() calls
I20260806 09:31:47.140574 7951 tablet_bootstrap.cc:492] T a1824279d6d444c5a53ea019c5a22073 P 8a2d3a09b6b0405a850828ac88d2e5e2: No bootstrap required, opened a new log
I20260806 09:31:47.140810 7949 tablet_bootstrap.cc:492] T a1824279d6d444c5a53ea019c5a22073 P 375a829a81c14acc8fb8f0c67661d7f7: No bootstrap required, opened a new log
I20260806 09:31:47.140887 7951 ts_tablet_manager.cc:1397] T a1824279d6d444c5a53ea019c5a22073 P 8a2d3a09b6b0405a850828ac88d2e5e2: Time spent bootstrapping tablet: real 0.018s user 0.008s sys 0.000s
I20260806 09:31:47.141227 7949 ts_tablet_manager.cc:1397] T a1824279d6d444c5a53ea019c5a22073 P 375a829a81c14acc8fb8f0c67661d7f7: Time spent bootstrapping tablet: real 0.038s user 0.006s sys 0.009s
I20260806 09:31:47.144680 7954 tablet_bootstrap.cc:492] T a1824279d6d444c5a53ea019c5a22073 P 66040518f530435183215ce1112bd084: Bootstrap starting.
I20260806 09:31:47.151183 7954 tablet_bootstrap.cc:654] T a1824279d6d444c5a53ea019c5a22073 P 66040518f530435183215ce1112bd084: Neither blocks nor log segments found. Creating new log.
I20260806 09:31:47.160550 7954 log.cc:824] T a1824279d6d444c5a53ea019c5a22073 P 66040518f530435183215ce1112bd084: Log is configured to *not* fsync() on all Append() calls
I20260806 09:31:47.162278 7949 raft_consensus.cc:348] T a1824279d6d444c5a53ea019c5a22073 P 375a829a81c14acc8fb8f0c67661d7f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "375a829a81c14acc8fb8f0c67661d7f7" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 34637 } } peers { permanent_uuid: "8a2d3a09b6b0405a850828ac88d2e5e2" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 44723 } } peers { permanent_uuid: "66040518f530435183215ce1112bd084" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33547 } }
I20260806 09:31:47.163543 7949 raft_consensus.cc:374] T a1824279d6d444c5a53ea019c5a22073 P 375a829a81c14acc8fb8f0c67661d7f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:31:47.163452 7951 raft_consensus.cc:348] T a1824279d6d444c5a53ea019c5a22073 P 8a2d3a09b6b0405a850828ac88d2e5e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "375a829a81c14acc8fb8f0c67661d7f7" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 34637 } } peers { permanent_uuid: "8a2d3a09b6b0405a850828ac88d2e5e2" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 44723 } } peers { permanent_uuid: "66040518f530435183215ce1112bd084" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33547 } }
I20260806 09:31:47.163908 7951 raft_consensus.cc:374] T a1824279d6d444c5a53ea019c5a22073 P 8a2d3a09b6b0405a850828ac88d2e5e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:31:47.164052 7951 raft_consensus.cc:729] T a1824279d6d444c5a53ea019c5a22073 P 8a2d3a09b6b0405a850828ac88d2e5e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a2d3a09b6b0405a850828ac88d2e5e2, State: Initialized, Role: FOLLOWER
I20260806 09:31:47.164105 7949 raft_consensus.cc:729] T a1824279d6d444c5a53ea019c5a22073 P 375a829a81c14acc8fb8f0c67661d7f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 375a829a81c14acc8fb8f0c67661d7f7, State: Initialized, Role: FOLLOWER
I20260806 09:31:47.164873 7951 consensus_queue.cc:260] T a1824279d6d444c5a53ea019c5a22073 P 8a2d3a09b6b0405a850828ac88d2e5e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "375a829a81c14acc8fb8f0c67661d7f7" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 34637 } } peers { permanent_uuid: "8a2d3a09b6b0405a850828ac88d2e5e2" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 44723 } } peers { permanent_uuid: "66040518f530435183215ce1112bd084" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33547 } }
I20260806 09:31:47.165576 7949 consensus_queue.cc:260] T a1824279d6d444c5a53ea019c5a22073 P 375a829a81c14acc8fb8f0c67661d7f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "375a829a81c14acc8fb8f0c67661d7f7" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 34637 } } peers { permanent_uuid: "8a2d3a09b6b0405a850828ac88d2e5e2" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 44723 } } peers { permanent_uuid: "66040518f530435183215ce1112bd084" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33547 } }
I20260806 09:31:47.169975 7911 heartbeater.cc:503] Master 127.20.113.62:40899 was elected leader, sending a full tablet report...
I20260806 09:31:47.172523 7954 tablet_bootstrap.cc:492] T a1824279d6d444c5a53ea019c5a22073 P 66040518f530435183215ce1112bd084: No bootstrap required, opened a new log
I20260806 09:31:47.172889 7954 ts_tablet_manager.cc:1397] T a1824279d6d444c5a53ea019c5a22073 P 66040518f530435183215ce1112bd084: Time spent bootstrapping tablet: real 0.028s user 0.002s sys 0.009s
I20260806 09:31:47.183643 7949 ts_tablet_manager.cc:1428] T a1824279d6d444c5a53ea019c5a22073 P 375a829a81c14acc8fb8f0c67661d7f7: Time spent starting tablet: real 0.042s user 0.010s sys 0.004s
I20260806 09:31:47.185216 7714 heartbeater.cc:503] Master 127.20.113.62:40899 was elected leader, sending a full tablet report...
I20260806 09:31:47.185722 7951 ts_tablet_manager.cc:1428] T a1824279d6d444c5a53ea019c5a22073 P 8a2d3a09b6b0405a850828ac88d2e5e2: Time spent starting tablet: real 0.045s user 0.014s sys 0.011s
I20260806 09:31:47.187026 7954 raft_consensus.cc:348] T a1824279d6d444c5a53ea019c5a22073 P 66040518f530435183215ce1112bd084 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "375a829a81c14acc8fb8f0c67661d7f7" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 34637 } } peers { permanent_uuid: "8a2d3a09b6b0405a850828ac88d2e5e2" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 44723 } } peers { permanent_uuid: "66040518f530435183215ce1112bd084" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33547 } }
I20260806 09:31:47.187515 7954 raft_consensus.cc:374] T a1824279d6d444c5a53ea019c5a22073 P 66040518f530435183215ce1112bd084 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:31:47.187628 7954 raft_consensus.cc:729] T a1824279d6d444c5a53ea019c5a22073 P 66040518f530435183215ce1112bd084 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66040518f530435183215ce1112bd084, State: Initialized, Role: FOLLOWER
I20260806 09:31:47.188282 7954 consensus_queue.cc:260] T a1824279d6d444c5a53ea019c5a22073 P 66040518f530435183215ce1112bd084 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "375a829a81c14acc8fb8f0c67661d7f7" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 34637 } } peers { permanent_uuid: "8a2d3a09b6b0405a850828ac88d2e5e2" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 44723 } } peers { permanent_uuid: "66040518f530435183215ce1112bd084" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33547 } }
I20260806 09:31:47.192291 7491 heartbeater.cc:503] Master 127.20.113.62:40899 was elected leader, sending a full tablet report...
I20260806 09:31:47.193081 7954 ts_tablet_manager.cc:1428] T a1824279d6d444c5a53ea019c5a22073 P 66040518f530435183215ce1112bd084: Time spent starting tablet: real 0.020s user 0.017s sys 0.000s
W20260806 09:31:47.326587 7912 tablet.cc:2369] T a1824279d6d444c5a53ea019c5a22073 P 375a829a81c14acc8fb8f0c67661d7f7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 09:31:47.385649 7493 tablet.cc:2369] T a1824279d6d444c5a53ea019c5a22073 P 66040518f530435183215ce1112bd084: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 09:31:47.392897 7715 tablet.cc:2369] T a1824279d6d444c5a53ea019c5a22073 P 8a2d3a09b6b0405a850828ac88d2e5e2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 09:31:47.569592 7964 raft_consensus.cc:482] T a1824279d6d444c5a53ea019c5a22073 P 66040518f530435183215ce1112bd084 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 09:31:47.570001 7964 raft_consensus.cc:504] T a1824279d6d444c5a53ea019c5a22073 P 66040518f530435183215ce1112bd084 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "375a829a81c14acc8fb8f0c67661d7f7" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 34637 } } peers { permanent_uuid: "8a2d3a09b6b0405a850828ac88d2e5e2" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 44723 } } peers { permanent_uuid: "66040518f530435183215ce1112bd084" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33547 } }
I20260806 09:31:47.571705 7964 leader_election.cc:288] T a1824279d6d444c5a53ea019c5a22073 P 66040518f530435183215ce1112bd084 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 375a829a81c14acc8fb8f0c67661d7f7 (127.20.113.3:34637), 8a2d3a09b6b0405a850828ac88d2e5e2 (127.20.113.2:44723)
I20260806 09:31:47.580351 7944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:31:47.573100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.1:35209 (local address 127.20.113.2:44723)
0806 09:31:47.573278 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0806 09:31:47.573283 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:31:47.573677 (+ 394us) server_negotiation.cc:407] Connection header received
0806 09:31:47.573855 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:31:47.573858 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:31:47.573957 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:31:47.574065 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:31:47.574612 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:31:47.575521 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:31:47.577912 (+ 2391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:31:47.579139 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:31:47.579280 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:31:47.579346 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 09:31:47.579902 (+ 556us) server_negotiation.cc:299] Negotiation successful
0806 09:31:47.580035 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 09:31:47.580355 7972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:31:47.572137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.1:60021 (local address 127.20.113.3:34637)
0806 09:31:47.572669 (+ 532us) server_negotiation.cc:206] Beginning negotiation
0806 09:31:47.572675 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:31:47.572904 (+ 229us) server_negotiation.cc:407] Connection header received
0806 09:31:47.573343 (+ 439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:31:47.573347 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:31:47.573404 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:31:47.573502 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:31:47.574194 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:31:47.575240 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:31:47.577847 (+ 2607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:31:47.579091 (+ 1244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:31:47.579272 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:31:47.579346 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 09:31:47.579892 (+ 546us) server_negotiation.cc:299] Negotiation successful
0806 09:31:47.580039 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":276,"threads_started":1}
I20260806 09:31:47.580701 7973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:31:47.572135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.3:34637 (local address 127.20.113.1:60021)
0806 09:31:47.572785 (+ 650us) negotiation.cc:107] Waiting for socket to connect
0806 09:31:47.572804 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 09:31:47.572993 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:31:47.573572 (+ 579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:31:47.573598 (+ 26us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:31:47.573637 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:31:47.574019 (+ 382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:31:47.574052 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:31:47.575362 (+ 1310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:31:47.575366 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:31:47.577656 (+ 2290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:31:47.577673 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:31:47.579420 (+ 1747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:31:47.579428 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:31:47.579682 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:31:47.579743 (+ 61us) client_negotiation.cc:769] Sending connection context
0806 09:31:47.579839 (+ 96us) client_negotiation.cc:240] Negotiation successful
0806 09:31:47.579990 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":533,"mutex_wait_us":75,"thread_start_us":350,"threads_started":1}
I20260806 09:31:47.581027 7974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:31:47.572917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.2:44723 (local address 127.20.113.1:35209)
0806 09:31:47.573540 (+ 623us) negotiation.cc:107] Waiting for socket to connect
0806 09:31:47.573561 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 09:31:47.573704 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:31:47.574097 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:31:47.574104 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:31:47.574134 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:31:47.574437 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:31:47.574445 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:31:47.575662 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:31:47.575671 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:31:47.577759 (+ 2088us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:31:47.577773 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:31:47.579436 (+ 1663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:31:47.579441 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:31:47.579668 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:31:47.579740 (+ 72us) client_negotiation.cc:769] Sending connection context
0806 09:31:47.579839 (+ 99us) client_negotiation.cc:240] Negotiation successful
0806 09:31:47.579990 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":466,"mutex_wait_us":62,"thread_start_us":295,"threads_started":1}
I20260806 09:31:47.581923 7848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1824279d6d444c5a53ea019c5a22073" candidate_uuid: "66040518f530435183215ce1112bd084" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "375a829a81c14acc8fb8f0c67661d7f7" is_pre_election: true
I20260806 09:31:47.581961 7644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1824279d6d444c5a53ea019c5a22073" candidate_uuid: "66040518f530435183215ce1112bd084" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a2d3a09b6b0405a850828ac88d2e5e2" is_pre_election: true
I20260806 09:31:47.582468 7848 raft_consensus.cc:2454] T a1824279d6d444c5a53ea019c5a22073 P 375a829a81c14acc8fb8f0c67661d7f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 66040518f530435183215ce1112bd084 in term 0.
I20260806 09:31:47.582491 7644 raft_consensus.cc:2454] T a1824279d6d444c5a53ea019c5a22073 P 8a2d3a09b6b0405a850828ac88d2e5e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 66040518f530435183215ce1112bd084 in term 0.
I20260806 09:31:47.583218 7331 leader_election.cc:302] T a1824279d6d444c5a53ea019c5a22073 P 66040518f530435183215ce1112bd084 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 375a829a81c14acc8fb8f0c67661d7f7, 66040518f530435183215ce1112bd084; no voters:
I20260806 09:31:47.583719 7964 raft_consensus.cc:2781] T a1824279d6d444c5a53ea019c5a22073 P 66040518f530435183215ce1112bd084 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 09:31:47.583923 7964 raft_consensus.cc:482] T a1824279d6d444c5a53ea019c5a22073 P 66040518f530435183215ce1112bd084 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 09:31:47.584062 7964 raft_consensus.cc:3037] T a1824279d6d444c5a53ea019c5a22073 P 66040518f530435183215ce1112bd084 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:31:47.586680 7964 raft_consensus.cc:504] T a1824279d6d444c5a53ea019c5a22073 P 66040518f530435183215ce1112bd084 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "375a829a81c14acc8fb8f0c67661d7f7" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 34637 } } peers { permanent_uuid: "8a2d3a09b6b0405a850828ac88d2e5e2" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 44723 } } peers { permanent_uuid: "66040518f530435183215ce1112bd084" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33547 } }
I20260806 09:31:47.587342 7964 leader_election.cc:288] T a1824279d6d444c5a53ea019c5a22073 P 66040518f530435183215ce1112bd084 [CANDIDATE]: Term 1 election: Requested vote from peers 375a829a81c14acc8fb8f0c67661d7f7 (127.20.113.3:34637), 8a2d3a09b6b0405a850828ac88d2e5e2 (127.20.113.2:44723)
I20260806 09:31:47.587919 7848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1824279d6d444c5a53ea019c5a22073" candidate_uuid: "66040518f530435183215ce1112bd084" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "375a829a81c14acc8fb8f0c67661d7f7"
I20260806 09:31:47.587980 7644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1824279d6d444c5a53ea019c5a22073" candidate_uuid: "66040518f530435183215ce1112bd084" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a2d3a09b6b0405a850828ac88d2e5e2"
I20260806 09:31:47.588157 7848 raft_consensus.cc:3037] T a1824279d6d444c5a53ea019c5a22073 P 375a829a81c14acc8fb8f0c67661d7f7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:31:47.588212 7644 raft_consensus.cc:3037] T a1824279d6d444c5a53ea019c5a22073 P 8a2d3a09b6b0405a850828ac88d2e5e2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:31:47.590612 7848 raft_consensus.cc:2454] T a1824279d6d444c5a53ea019c5a22073 P 375a829a81c14acc8fb8f0c67661d7f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66040518f530435183215ce1112bd084 in term 1.
I20260806 09:31:47.590688 7644 raft_consensus.cc:2454] T a1824279d6d444c5a53ea019c5a22073 P 8a2d3a09b6b0405a850828ac88d2e5e2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66040518f530435183215ce1112bd084 in term 1.
I20260806 09:31:47.591161 7331 leader_election.cc:302] T a1824279d6d444c5a53ea019c5a22073 P 66040518f530435183215ce1112bd084 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66040518f530435183215ce1112bd084, 8a2d3a09b6b0405a850828ac88d2e5e2; no voters:
I20260806 09:31:47.591514 7964 raft_consensus.cc:2781] T a1824279d6d444c5a53ea019c5a22073 P 66040518f530435183215ce1112bd084 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:31:47.591961 7964 raft_consensus.cc:686] T a1824279d6d444c5a53ea019c5a22073 P 66040518f530435183215ce1112bd084 [term 1 LEADER]: Becoming Leader. State: Replica: 66040518f530435183215ce1112bd084, State: Running, Role: LEADER
I20260806 09:31:47.592559 7964 consensus_queue.cc:237] T a1824279d6d444c5a53ea019c5a22073 P 66040518f530435183215ce1112bd084 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "375a829a81c14acc8fb8f0c67661d7f7" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 34637 } } peers { permanent_uuid: "8a2d3a09b6b0405a850828ac88d2e5e2" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 44723 } } peers { permanent_uuid: "66040518f530435183215ce1112bd084" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33547 } }
I20260806 09:31:47.597645 7207 catalog_manager.cc:5495] T a1824279d6d444c5a53ea019c5a22073 P 66040518f530435183215ce1112bd084 reported cstate change: term changed from 0 to 1, leader changed from <none> to 66040518f530435183215ce1112bd084 (127.20.113.1). New cstate: current_term: 1 leader_uuid: "66040518f530435183215ce1112bd084" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "375a829a81c14acc8fb8f0c67661d7f7" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 34637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a2d3a09b6b0405a850828ac88d2e5e2" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 44723 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "66040518f530435183215ce1112bd084" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33547 } health_report { overall_health: HEALTHY } } }
I20260806 09:31:47.630357 7985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:31:47.624033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49992 (local address 127.20.113.1:33547)
0806 09:31:47.624759 (+ 726us) server_negotiation.cc:206] Beginning negotiation
0806 09:31:47.624766 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:31:47.624785 (+ 19us) server_negotiation.cc:407] Connection header received
0806 09:31:47.624926 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:31:47.624930 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:31:47.625063 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:31:47.625247 (+ 184us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 09:31:47.625726 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:31:47.626792 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:31:47.628131 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:31:47.628616 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:31:47.628715 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:31:47.629167 (+ 452us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 09:31:47.629753 (+ 586us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 09:31:47.629874 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0806 09:31:47.629971 (+ 97us) server_negotiation.cc:299] Negotiation successful
0806 09:31:47.630037 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":570,"thread_start_us":438,"threads_started":1}
I20260806 09:31:47.639127 7644 raft_consensus.cc:1261] T a1824279d6d444c5a53ea019c5a22073 P 8a2d3a09b6b0405a850828ac88d2e5e2 [term 1 FOLLOWER]: Refusing update from remote peer 66040518f530435183215ce1112bd084: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 09:31:47.639322 7848 raft_consensus.cc:1261] T a1824279d6d444c5a53ea019c5a22073 P 375a829a81c14acc8fb8f0c67661d7f7 [term 1 FOLLOWER]: Refusing update from remote peer 66040518f530435183215ce1112bd084: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 09:31:47.640209 7975 consensus_queue.cc:1035] T a1824279d6d444c5a53ea019c5a22073 P 66040518f530435183215ce1112bd084 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a2d3a09b6b0405a850828ac88d2e5e2" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 44723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 09:31:47.640676 7964 consensus_queue.cc:1035] T a1824279d6d444c5a53ea019c5a22073 P 66040518f530435183215ce1112bd084 [LEADER]: Connected to new peer: Peer: permanent_uuid: "375a829a81c14acc8fb8f0c67661d7f7" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 34637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 09:31:47.652185 7986 mvcc.cc:205] Tried to move back new op lower bound from 7315491666479800320 to 7315491666303250432. Current Snapshot: MvccSnapshot[applied={T|T < 7315491666479800320}]
I20260806 09:31:47.667479 7207 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57454:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 09:31:47.667740 7207 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57454:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 09:31:47.671420 7207 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4acecc18510e46919fbd36bbdbe95ed7: Sending DeleteTablet for 3 replicas of tablet a1824279d6d444c5a53ea019c5a22073
I20260806 09:31:47.672595 7615 tablet_service.cc:1514] Processing DeleteTablet for tablet a1824279d6d444c5a53ea019c5a22073 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:31:47 UTC) from {username='kudu', principal='kudu/127.20.113.62@KRBTEST.COM'} at 127.0.0.1:56886
I20260806 09:31:47.672621 7395 tablet_service.cc:1514] Processing DeleteTablet for tablet a1824279d6d444c5a53ea019c5a22073 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:31:47 UTC) from {username='kudu', principal='kudu/127.20.113.62@KRBTEST.COM'} at 127.0.0.1:49990
I20260806 09:31:47.673060 7816 tablet_service.cc:1514] Processing DeleteTablet for tablet a1824279d6d444c5a53ea019c5a22073 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:31:47 UTC) from {username='kudu', principal='kudu/127.20.113.62@KRBTEST.COM'} at 127.0.0.1:44096
I20260806 09:31:47.673671 7997 tablet_replica.cc:330] T a1824279d6d444c5a53ea019c5a22073 P 8a2d3a09b6b0405a850828ac88d2e5e2: stopping tablet replica
I20260806 09:31:47.674177 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 7259
I20260806 09:31:47.674321 7997 raft_consensus.cc:2229] T a1824279d6d444c5a53ea019c5a22073 P 8a2d3a09b6b0405a850828ac88d2e5e2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 09:31:47.675177 7997 raft_consensus.cc:2258] T a1824279d6d444c5a53ea019c5a22073 P 8a2d3a09b6b0405a850828ac88d2e5e2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 09:31:47.677578 7997 ts_tablet_manager.cc:1905] T a1824279d6d444c5a53ea019c5a22073 P 8a2d3a09b6b0405a850828ac88d2e5e2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 09:31:47.685199 7997 ts_tablet_manager.cc:1918] T a1824279d6d444c5a53ea019c5a22073 P 8a2d3a09b6b0405a850828ac88d2e5e2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 09:31:47.685475 7997 log.cc:1196] T a1824279d6d444c5a53ea019c5a22073 P 8a2d3a09b6b0405a850828ac88d2e5e2: Deleting WAL directory at /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ts-1/wal/wals/a1824279d6d444c5a53ea019c5a22073
I20260806 09:31:47.686317 7997 ts_tablet_manager.cc:1939] T a1824279d6d444c5a53ea019c5a22073 P 8a2d3a09b6b0405a850828ac88d2e5e2: Deleting consensus metadata
I20260806 09:31:47.687104 7998 tablet_replica.cc:330] T a1824279d6d444c5a53ea019c5a22073 P 66040518f530435183215ce1112bd084: stopping tablet replica
I20260806 09:31:47.687601 7998 raft_consensus.cc:2229] T a1824279d6d444c5a53ea019c5a22073 P 66040518f530435183215ce1112bd084 [term 1 LEADER]: Raft consensus shutting down.
I20260806 09:31:47.688411 7998 raft_consensus.cc:2258] T a1824279d6d444c5a53ea019c5a22073 P 66040518f530435183215ce1112bd084 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 09:31:47.688524 7183 catalog_manager.cc:4841] TS 8a2d3a09b6b0405a850828ac88d2e5e2 (127.20.113.2:44723): tablet a1824279d6d444c5a53ea019c5a22073 (table test-table [id=ea6a40176ab841dbb2b1fd533ca3428c]) successfully deleted
I20260806 09:31:47.690670 7998 ts_tablet_manager.cc:1905] T a1824279d6d444c5a53ea019c5a22073 P 66040518f530435183215ce1112bd084: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 09:31:47.695619 8000 tablet_replica.cc:330] T a1824279d6d444c5a53ea019c5a22073 P 375a829a81c14acc8fb8f0c67661d7f7: stopping tablet replica
I20260806 09:31:47.696142 8000 raft_consensus.cc:2229] T a1824279d6d444c5a53ea019c5a22073 P 375a829a81c14acc8fb8f0c67661d7f7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 09:31:47.696724 8000 raft_consensus.cc:2258] T a1824279d6d444c5a53ea019c5a22073 P 375a829a81c14acc8fb8f0c67661d7f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 09:31:47.698426 7998 ts_tablet_manager.cc:1918] T a1824279d6d444c5a53ea019c5a22073 P 66040518f530435183215ce1112bd084: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 09:31:47.698618 7998 log.cc:1196] T a1824279d6d444c5a53ea019c5a22073 P 66040518f530435183215ce1112bd084: Deleting WAL directory at /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ts-0/wal/wals/a1824279d6d444c5a53ea019c5a22073
I20260806 09:31:47.699198 7998 ts_tablet_manager.cc:1939] T a1824279d6d444c5a53ea019c5a22073 P 66040518f530435183215ce1112bd084: Deleting consensus metadata
I20260806 09:31:47.700222 8000 ts_tablet_manager.cc:1905] T a1824279d6d444c5a53ea019c5a22073 P 375a829a81c14acc8fb8f0c67661d7f7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 09:31:47.700891 7184 catalog_manager.cc:4841] TS 66040518f530435183215ce1112bd084 (127.20.113.1:33547): tablet a1824279d6d444c5a53ea019c5a22073 (table test-table [id=ea6a40176ab841dbb2b1fd533ca3428c]) successfully deleted
I20260806 09:31:47.708580 8000 ts_tablet_manager.cc:1918] T a1824279d6d444c5a53ea019c5a22073 P 375a829a81c14acc8fb8f0c67661d7f7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 09:31:47.708781 8000 log.cc:1196] T a1824279d6d444c5a53ea019c5a22073 P 375a829a81c14acc8fb8f0c67661d7f7: Deleting WAL directory at /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786008495763054-20932-0/minicluster-data/ts-2/wal/wals/a1824279d6d444c5a53ea019c5a22073
I20260806 09:31:47.709334 8000 ts_tablet_manager.cc:1939] T a1824279d6d444c5a53ea019c5a22073 P 375a829a81c14acc8fb8f0c67661d7f7: Deleting consensus metadata
I20260806 09:31:47.710217 7985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:31:47.675090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50008 (local address 127.20.113.1:33547)
0806 09:31:47.675265 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0806 09:31:47.675272 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:31:47.692875 (+ 17603us) server_negotiation.cc:407] Connection header received
0806 09:31:47.693091 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:31:47.693099 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:31:47.693207 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:31:47.693319 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:31:47.695328 (+ 2009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:31:47.696646 (+ 1318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:31:47.698823 (+ 2177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:31:47.699542 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:31:47.699672 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:31:47.703377 (+ 3705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:31:47.703389 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:31:47.703402 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:31:47.703444 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:31:47.706953 (+ 3509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:31:47.707686 (+ 733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:31:47.707698 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:31:47.707700 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:31:47.707870 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:31:47.708298 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:31:47.708342 (+ 44us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:31:47.708345 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:31:47.708885 (+ 540us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:31:47.709200 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0806 09:31:47.709549 (+ 349us) server_negotiation.cc:299] Negotiation successful
0806 09:31:47.709817 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 09:31:47.711154 7484 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:31:47.711277 7183 catalog_manager.cc:4841] TS 375a829a81c14acc8fb8f0c67661d7f7 (127.20.113.3:34637): tablet a1824279d6d444c5a53ea019c5a22073 (table test-table [id=ea6a40176ab841dbb2b1fd533ca3428c]) successfully deleted
I20260806 09:31:47.811363 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 7259
I20260806 09:31:47.837865 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 7497
I20260806 09:31:47.852145 7944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:31:47.838666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56890 (local address 127.20.113.2:44723)
0806 09:31:47.838843 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0806 09:31:47.838850 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:31:47.838890 (+ 40us) server_negotiation.cc:407] Connection header received
0806 09:31:47.839164 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:31:47.839168 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:31:47.839256 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:31:47.839378 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:31:47.840934 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:31:47.841700 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:31:47.843027 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:31:47.843393 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:31:47.843490 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:31:47.846478 (+ 2988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:31:47.846491 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:31:47.846505 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:31:47.846535 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:31:47.849493 (+ 2958us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:31:47.850148 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:31:47.850155 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:31:47.850157 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:31:47.850278 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:31:47.850648 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:31:47.850653 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:31:47.850656 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:31:47.850989 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:31:47.851278 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0806 09:31:47.851621 (+ 343us) server_negotiation.cc:299] Negotiation successful
0806 09:31:47.851828 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 09:31:47.852833 7708 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:31:47.938390 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 7497
I20260806 09:31:47.962917 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 7718
I20260806 09:31:47.976579 7972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:31:47.963729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44112 (local address 127.20.113.3:34637)
0806 09:31:47.963926 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0806 09:31:47.963931 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:31:47.963962 (+ 31us) server_negotiation.cc:407] Connection header received
0806 09:31:47.964019 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:31:47.964022 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:31:47.964083 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:31:47.964163 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:31:47.965498 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:31:47.966161 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:31:47.967386 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:31:47.967785 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:31:47.967873 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:31:47.970777 (+ 2904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:31:47.970792 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:31:47.970808 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:31:47.970840 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:31:47.973676 (+ 2836us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:31:47.974476 (+ 800us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:31:47.974484 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:31:47.974486 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:31:47.974643 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:31:47.975001 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:31:47.975005 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:31:47.975007 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:31:47.975363 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:31:47.975714 (+ 351us) server_negotiation.cc:1035] Waiting for connection context
0806 09:31:47.975946 (+ 232us) server_negotiation.cc:299] Negotiation successful
0806 09:31:47.976122 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 09:31:47.977296 7903 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:31:48.064349 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 7718
I20260806 09:31:48.090703 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 7155
I20260806 09:31:48.091950 7248 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:31:48.203555 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 7155
I20260806 09:31:48.240985 20932 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260806 09:31:48.585919 20932 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260806 09:31:48.586153 20932 mini_ranger.cc:66] Stopping Ranger...
I20260806 09:31:48.940853 20932 mini_ranger.cc:68] Stopped Ranger
2026-08-06 09:31:48.941 UTC [5820] LOG: received smart shutdown request
2026-08-06 09:31:48.944 UTC [5820] LOG: background worker "logical replication launcher" (PID 5829) exited with exit code 1
2026-08-06 09:31:48.944 UTC [5824] LOG: shutting down
2026-08-06 09:31:49.094 UTC [5820] LOG: database system is shut down
2026-08-06T09:31:49Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (54012 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786008495763054-20932-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 09:31:49 dist-test-slave-45dx krb5kdc[8023](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 09:31:49 dist-test-slave-45dx krb5kdc[8023](info): set up 2 sockets
Aug 06 09:31:49 dist-test-slave-45dx krb5kdc[8023](info): commencing operation
krb5kdc: starting...
W20260806 09:31:50.547585 20932 mini_kdc.cc:121] Time spent starting KDC: real 1.272s user 0.000s sys 0.049s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786008495763054-20932-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786008495763054-20932-0/krb5kdc/test-user.keytab.
Aug 06 09:31:50 dist-test-slave-45dx krb5kdc[8023](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008710, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T09:31:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T09:31:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.113.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.113.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.113.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab.
Entry for principal kudu/127.20.113.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab.
I20260806 09:31:50.851986 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786008495763054-20932-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786008495763054-20932-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786008495763054-20932-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.113.62:41515
--webserver_interface=127.20.113.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab
--principal=kudu/127.20.113.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:39499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.113.62:41515,127.20.113.61:44499,127.20.113.60:34865
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:31:51.198733 8039 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:31:51.199096 8039 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:31:51.199270 8039 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:31:51.206486 8039 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 09:31:51.206634 8039 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:31:51.206764 8039 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:31:51.206877 8039 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 09:31:51.206969 8039 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 09:31:51.217103 8039 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:39499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786008495763054-20932-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786008495763054-20932-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.113.62:41515,127.20.113.61:44499,127.20.113.60:34865
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab
--principal=kudu/127.20.113.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.113.62:41515
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.113.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786008495763054-20932-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:31:51.218600 8039 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:31:51.220026 8039 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:31:51.232803 8045 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:31:51.232894 8049 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:31:51.232856 8047 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:31:51.233742 8039 server_base.cc:1034] running on GCE node
I20260806 09:31:51.234473 8039 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:31:51.236076 8039 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:31:51.237346 8039 hybrid_clock.cc:648] HybridClock initialized: now 1786008711237273 us; error 44 us; skew 500 ppm
I20260806 09:31:51.237733 8039 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:31:51 dist-test-slave-45dx krb5kdc[8023](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008711, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:31:51.247453 8039 init.cc:377] Logged in from keytab as kudu/127.20.113.62@KRBTEST.COM (short username kudu)
I20260806 09:31:51.249094 8039 webserver.cc:460] Webserver started at http://127.20.113.62:38533/ using document root <none> and password file <none>
I20260806 09:31:51.249794 8039 fs_manager.cc:359] Metadata directory not provided
I20260806 09:31:51.249943 8039 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:31:51.250279 8039 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:31:51.253351 8039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786008495763054-20932-0/minicluster-data/master-0/data/instance:
uuid: "51a55182913d414b8cb7cf5eef9e1f3d"
format_stamp: "Formatted at 2026-08-06 09:31:51 on dist-test-slave-45dx"
server_key: "e3bcae3617b2a75777ce165ff0453a8a"
server_key_iv: "70b8b17f7df822e623ba90b4891484b4"
server_key_version: "encryptionkey@0"
I20260806 09:31:51.254163 8039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786008495763054-20932-0/minicluster-data/master-0/wal/instance:
uuid: "51a55182913d414b8cb7cf5eef9e1f3d"
format_stamp: "Formatted at 2026-08-06 09:31:51 on dist-test-slave-45dx"
server_key: "e3bcae3617b2a75777ce165ff0453a8a"
server_key_iv: "70b8b17f7df822e623ba90b4891484b4"
server_key_version: "encryptionkey@0"
I20260806 09:31:51.259222 8039 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 09:31:51.262763 8061 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:31:51.264144 8039 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260806 09:31:51.264377 8039 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786008495763054-20932-0/minicluster-data/master-0/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786008495763054-20932-0/minicluster-data/master-0/wal
uuid: "51a55182913d414b8cb7cf5eef9e1f3d"
format_stamp: "Formatted at 2026-08-06 09:31:51 on dist-test-slave-45dx"
server_key: "e3bcae3617b2a75777ce165ff0453a8a"
server_key_iv: "70b8b17f7df822e623ba90b4891484b4"
server_key_version: "encryptionkey@0"
I20260806 09:31:51.264580 8039 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786008495763054-20932-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786008495763054-20932-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786008495763054-20932-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:31:51.275337 8039 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:31:51.279085 8039 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:31:51.279381 8039 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:31:51.298347 8039 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.62:41515
I20260806 09:31:51.298383 8135 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.62:41515 every 8 connection(s)
I20260806 09:31:51.299685 8039 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
I20260806 09:31:51.301364 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 8039
I20260806 09:31:51.301610 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786008495763054-20932-0/minicluster-data/master-0/wal/instance
I20260806 09:31:51.302316 20932 external_mini_cluster.cc:1428] Setting key c996841c3d988d7d5de43c75da6f10a0
I20260806 09:31:51.303418 8137 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:31:51.309641 8137 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.113.62" port: 41515 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 09:31:51 dist-test-slave-45dx krb5kdc[8023](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008711, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.62@KRBTEST.COM for kudu/127.20.113.62@KRBTEST.COM
I20260806 09:31:51.331346 8141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:31:51.312660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:41515 (local address 127.0.0.1:52636)
0806 09:31:51.313785 (+ 1125us) negotiation.cc:107] Waiting for socket to connect
0806 09:31:51.313818 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 09:31:51.315531 (+ 1713us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:31:51.316297 (+ 766us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:31:51.316324 (+ 27us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:31:51.316986 (+ 662us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:31:51.317291 (+ 305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:31:51.317326 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:31:51.318861 (+ 1535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:31:51.318865 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:31:51.319896 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:31:51.319917 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:31:51.320849 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:31:51.320856 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:31:51.321047 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:31:51.321719 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:31:51.321740 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:31:51.323840 (+ 2100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:31:51.327072 (+ 3232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:31:51.327084 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:31:51.327096 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:31:51.327488 (+ 392us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:31:51.327886 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:31:51.327893 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:31:51.327899 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:31:51.328064 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:31:51.328865 (+ 801us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:31:51.328878 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:31:51.329096 (+ 218us) client_negotiation.cc:769] Sending connection context
0806 09:31:51.329334 (+ 238us) client_negotiation.cc:240] Negotiation successful
0806 09:31:51.329994 (+ 660us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":926,"mutex_wait_us":205,"thread_start_us":708,"threads_started":1}
I20260806 09:31:51.331326 8139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:31:51.312645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52636 (local address 127.20.113.62:41515)
0806 09:31:51.313504 (+ 859us) server_negotiation.cc:206] Beginning negotiation
0806 09:31:51.313525 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 09:31:51.313942 (+ 417us) server_negotiation.cc:407] Connection header received
0806 09:31:51.315869 (+ 1927us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:31:51.315875 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:31:51.316076 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:31:51.316267 (+ 191us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:31:51.317515 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:31:51.318693 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:31:51.320048 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:31:51.320626 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:31:51.320831 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:31:51.324058 (+ 3227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:31:51.324065 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:31:51.324076 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:31:51.324103 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:31:51.326940 (+ 2837us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:31:51.327618 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:31:51.327627 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:31:51.327632 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:31:51.327777 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:31:51.328171 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:31:51.328179 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:31:51.328182 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:31:51.328730 (+ 548us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:31:51.328944 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0806 09:31:51.329459 (+ 515us) server_negotiation.cc:299] Negotiation successful
0806 09:31:51.329857 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":39,"server-negotiator.queue_time_us":574,"thread_start_us":370,"threads_started":1}
WARNING: no policy specified for kudu/127.20.113.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.113.61@KRBTEST.COM" created.
I20260806 09:31:51.334594 8137 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.113.61" port: 44499 } has no permanent_uuid. Determining permanent_uuid...
I20260806 09:31:51.335666 8141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:31:51.335271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.61:44499 (local address 127.0.0.1:44186)
0806 09:31:51.335439 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0806 09:31:51.335554 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.61:44499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260806 09:31:51.335949 8063 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.113.61:44499: connect: Connection refused (error 111)
I20260806 09:31:51.337615 8141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:31:51.337232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.61:44499 (local address 127.0.0.1:44200)
0806 09:31:51.337418 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0806 09:31:51.337484 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.61:44499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260806 09:31:51.337795 8137 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.113.61:44499: Network error: Client connection negotiation failed: client connection to 127.20.113.61:44499: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.113.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.61.keytab.
Entry for principal kudu/127.20.113.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.61.keytab.
I20260806 09:31:51.371723 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786008495763054-20932-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786008495763054-20932-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786008495763054-20932-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786008495763054-20932-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.113.61:44499
--webserver_interface=127.20.113.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.61.keytab
--principal=kudu/127.20.113.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:39499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.113.62:41515,127.20.113.61:44499,127.20.113.60:34865
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 09:31:51.387346 8137 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.113.61" port: 44499 } attempt: 1
I20260806 09:31:51.388579 8141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:31:51.388072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.61:44499 (local address 127.0.0.1:44214)
0806 09:31:51.388266 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0806 09:31:51.388431 (+ 165us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.61:44499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260806 09:31:51.389535 8141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:31:51.389193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.61:44499 (local address 127.0.0.1:44230)
0806 09:31:51.389335 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0806 09:31:51.389411 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.61:44499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260806 09:31:51.389707 8137 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.113.61:44499: Network error: Client connection negotiation failed: client connection to 127.20.113.61:44499: connect: Connection refused (error 111)
I20260806 09:31:51.457994 8137 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.113.61" port: 44499 } attempt: 2
I20260806 09:31:51.459262 8141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:31:51.458722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.61:44499 (local address 127.0.0.1:44234)
0806 09:31:51.459015 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0806 09:31:51.459113 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.61:44499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260806 09:31:51.460397 8141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:31:51.459960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.61:44499 (local address 127.0.0.1:44238)
0806 09:31:51.460112 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0806 09:31:51.460200 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.61:44499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260806 09:31:51.460582 8137 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.113.61:44499: Network error: Client connection negotiation failed: client connection to 127.20.113.61:44499: connect: Connection refused (error 111)
I20260806 09:31:51.551884 8137 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.113.61" port: 44499 } attempt: 3
I20260806 09:31:51.553211 8141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:31:51.552722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.61:44499 (local address 127.0.0.1:44244)
0806 09:31:51.552925 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0806 09:31:51.553049 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.61:44499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260806 09:31:51.554237 8141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:31:51.553874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.61:44499 (local address 127.0.0.1:44260)
0806 09:31:51.554020 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0806 09:31:51.554097 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.61:44499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260806 09:31:51.554457 8137 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.113.61:44499: Network error: Client connection negotiation failed: client connection to 127.20.113.61:44499: connect: Connection refused (error 111)
I20260806 09:31:51.697713 8137 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.113.61" port: 44499 } attempt: 4
I20260806 09:31:51.698963 8141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:31:51.698411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.61:44499 (local address 127.0.0.1:44270)
0806 09:31:51.698730 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0806 09:31:51.698818 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.61:44499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157}
I20260806 09:31:51.699972 8141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:31:51.699618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.61:44499 (local address 127.0.0.1:44272)
0806 09:31:51.699763 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0806 09:31:51.699855 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.61:44499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260806 09:31:51.700188 8137 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.113.61:44499: Network error: Client connection negotiation failed: client connection to 127.20.113.61:44499: connect: Connection refused (error 111)
W20260806 09:31:51.712437 8146 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:31:51.712764 8146 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:31:51.712939 8146 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:31:51.719967 8146 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 09:31:51.720101 8146 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:31:51.720209 8146 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:31:51.720336 8146 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 09:31:51.720407 8146 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 09:31:51.729993 8146 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:39499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786008495763054-20932-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786008495763054-20932-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.20.113.62:41515,127.20.113.61:44499,127.20.113.60:34865
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.61.keytab
--principal=kudu/127.20.113.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.113.61:44499
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786008495763054-20932-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.20.113.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786008495763054-20932-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:31:51.731325 8146 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:31:51.732707 8146 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:31:51.745231 8154 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:31:51.745282 8155 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:31:51.745944 8159 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:31:51.746349 8146 server_base.cc:1034] running on GCE node
I20260806 09:31:51.747141 8146 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:31:51.748456 8146 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:31:51.749684 8146 hybrid_clock.cc:648] HybridClock initialized: now 1786008711749599 us; error 71 us; skew 500 ppm
I20260806 09:31:51.750027 8146 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:31:51 dist-test-slave-45dx krb5kdc[8023](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008711, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:31:51.753233 8146 init.cc:377] Logged in from keytab as kudu/127.20.113.61@KRBTEST.COM (short username kudu)
I20260806 09:31:51.754624 8146 webserver.cc:460] Webserver started at http://127.20.113.61:42759/ using document root <none> and password file <none>
I20260806 09:31:51.755300 8146 fs_manager.cc:359] Metadata directory not provided
I20260806 09:31:51.755457 8146 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:31:51.755800 8146 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:31:51.758852 8146 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786008495763054-20932-0/minicluster-data/master-1/data/instance:
uuid: "91ace7d8479a4eb8abca1d23fa091e5f"
format_stamp: "Formatted at 2026-08-06 09:31:51 on dist-test-slave-45dx"
server_key: "18d00e14882a833fe559a93a7dd792dc"
server_key_iv: "ef6c08a74b462a3744413b67595ef90a"
server_key_version: "encryptionkey@0"
I20260806 09:31:51.759698 8146 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786008495763054-20932-0/minicluster-data/master-1/wal/instance:
uuid: "91ace7d8479a4eb8abca1d23fa091e5f"
format_stamp: "Formatted at 2026-08-06 09:31:51 on dist-test-slave-45dx"
server_key: "18d00e14882a833fe559a93a7dd792dc"
server_key_iv: "ef6c08a74b462a3744413b67595ef90a"
server_key_version: "encryptionkey@0"
I20260806 09:31:51.764846 8146 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 09:31:51.768379 8170 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:31:51.769732 8146 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 09:31:51.769919 8146 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786008495763054-20932-0/minicluster-data/master-1/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786008495763054-20932-0/minicluster-data/master-1/wal
uuid: "91ace7d8479a4eb8abca1d23fa091e5f"
format_stamp: "Formatted at 2026-08-06 09:31:51 on dist-test-slave-45dx"
server_key: "18d00e14882a833fe559a93a7dd792dc"
server_key_iv: "ef6c08a74b462a3744413b67595ef90a"
server_key_version: "encryptionkey@0"
I20260806 09:31:51.770133 8146 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786008495763054-20932-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786008495763054-20932-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786008495763054-20932-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:31:51.795707 8146 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:31:51.800094 8146 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:31:51.800422 8146 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:31:51.818598 8146 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.61:44499
I20260806 09:31:51.818629 8233 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.61:44499 every 8 connection(s)
I20260806 09:31:51.819927 8146 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786008495763054-20932-0/minicluster-data/master-1/data/info.pb
I20260806 09:31:51.823608 8234 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:31:51.830013 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 8146
I20260806 09:31:51.830211 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786008495763054-20932-0/minicluster-data/master-1/wal/instance
I20260806 09:31:51.830681 20932 external_mini_cluster.cc:1428] Setting key 32fa243ea200a915cf73831057fdb8f6
I20260806 09:31:51.830178 8234 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.113.62" port: 41515 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 09:31:51 dist-test-slave-45dx krb5kdc[8023](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008711, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.61@KRBTEST.COM for kudu/127.20.113.62@KRBTEST.COM
I20260806 09:31:51.852803 8237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:31:51.832889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52652 (local address 127.20.113.62:41515)
0806 09:31:51.833362 (+ 473us) server_negotiation.cc:206] Beginning negotiation
0806 09:31:51.833367 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:31:51.834483 (+ 1116us) server_negotiation.cc:407] Connection header received
0806 09:31:51.836297 (+ 1814us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:31:51.836321 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:31:51.836417 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:31:51.836539 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:31:51.838007 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:31:51.838721 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:31:51.840430 (+ 1709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:31:51.840842 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:31:51.840981 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:31:51.844358 (+ 3377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:31:51.844363 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:31:51.844367 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:31:51.844415 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:31:51.848591 (+ 4176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:31:51.849616 (+ 1025us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:31:51.849622 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:31:51.849626 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:31:51.849808 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:31:51.850458 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:31:51.850466 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:31:51.850472 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:31:51.850867 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:31:51.851038 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0806 09:31:51.851996 (+ 958us) server_negotiation.cc:299] Negotiation successful
0806 09:31:51.852248 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":365,"thread_start_us":261,"threads_started":1}
I20260806 09:31:51.854137 8238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:31:51.833561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:41515 (local address 127.0.0.1:52652)
0806 09:31:51.834319 (+ 758us) negotiation.cc:107] Waiting for socket to connect
0806 09:31:51.834357 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 09:31:51.836032 (+ 1675us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:31:51.836785 (+ 753us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:31:51.836793 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:31:51.837289 (+ 496us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:31:51.837818 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:31:51.837835 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:31:51.838892 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:31:51.838896 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:31:51.840247 (+ 1351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:31:51.840268 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:31:51.841073 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:31:51.841081 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:31:51.841299 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:31:51.842002 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:31:51.842023 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:31:51.844123 (+ 2100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:31:51.848750 (+ 4627us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:31:51.848761 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:31:51.848779 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:31:51.849451 (+ 672us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:31:51.850038 (+ 587us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:31:51.850047 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:31:51.850054 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:31:51.850308 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:31:51.851038 (+ 730us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:31:51.851047 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:31:51.851504 (+ 457us) client_negotiation.cc:769] Sending connection context
0806 09:31:51.851863 (+ 359us) client_negotiation.cc:240] Negotiation successful
0806 09:31:51.852433 (+ 570us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":535,"thread_start_us":342,"threads_started":1}
I20260806 09:31:51.856755 8234 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.113.61" port: 44499 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.20.113.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.113.60@KRBTEST.COM" created.
Aug 06 09:31:51 dist-test-slave-45dx krb5kdc[8023](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008711, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.61@KRBTEST.COM for kudu/127.20.113.61@KRBTEST.COM
I20260806 09:31:51.878306 8238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:31:51.857568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.61:44499 (local address 127.0.0.1:44280)
0806 09:31:51.858085 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0806 09:31:51.858104 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 09:31:51.858227 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:31:51.858933 (+ 706us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:31:51.858939 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:31:51.859422 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:31:51.859616 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:31:51.859632 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:31:51.861939 (+ 2307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:31:51.861948 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:31:51.863381 (+ 1433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:31:51.863397 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:31:51.864801 (+ 1404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:31:51.864808 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:31:51.865023 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:31:51.867355 (+ 2332us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:31:51.867380 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:31:51.871466 (+ 4086us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:31:51.875202 (+ 3736us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:31:51.875208 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:31:51.875215 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:31:51.875581 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:31:51.875961 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:31:51.875966 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:31:51.875969 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:31:51.876113 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:31:51.877144 (+ 1031us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:31:51.877149 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:31:51.877322 (+ 173us) client_negotiation.cc:769] Sending connection context
0806 09:31:51.877488 (+ 166us) client_negotiation.cc:240] Negotiation successful
0806 09:31:51.877822 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":373}
I20260806 09:31:51.879599 8243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:31:51.857854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44280 (local address 127.20.113.61:44499)
0806 09:31:51.858558 (+ 704us) server_negotiation.cc:206] Beginning negotiation
0806 09:31:51.858564 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:31:51.858587 (+ 23us) server_negotiation.cc:407] Connection header received
0806 09:31:51.858626 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:31:51.858631 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:31:51.858788 (+ 157us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:31:51.858946 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:31:51.859903 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:31:51.861708 (+ 1805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:31:51.863550 (+ 1842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:31:51.864466 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:31:51.864588 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:31:51.871703 (+ 7115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:31:51.871716 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:31:51.871729 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:31:51.871756 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:31:51.875078 (+ 3322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:31:51.875711 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:31:51.875719 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:31:51.875723 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:31:51.875846 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:31:51.876224 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:31:51.876229 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:31:51.876232 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:31:51.876855 (+ 623us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:31:51.877071 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0806 09:31:51.878815 (+ 1744us) server_negotiation.cc:299] Negotiation successful
0806 09:31:51.879129 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":569,"thread_start_us":415,"threads_started":1}
I20260806 09:31:51.883240 8234 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.113.60" port: 34865 } has no permanent_uuid. Determining permanent_uuid...
I20260806 09:31:51.884970 8238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:31:51.884195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.60:34865 (local address 127.0.0.1:60534)
0806 09:31:51.884587 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0806 09:31:51.884722 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.60:34865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":232}
W20260806 09:31:51.885388 8174 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.113.60:34865: connect: Connection refused (error 111)
I20260806 09:31:51.887504 8238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:31:51.887043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.60:34865 (local address 127.0.0.1:60544)
0806 09:31:51.887243 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0806 09:31:51.887355 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.60:34865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260806 09:31:51.888200 8234 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.113.60:34865: Network error: Client connection negotiation failed: client connection to 127.20.113.60:34865: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.113.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.60.keytab.
Entry for principal kudu/127.20.113.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.60.keytab.
I20260806 09:31:51.913179 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786008495763054-20932-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786008495763054-20932-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786008495763054-20932-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786008495763054-20932-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.113.60:34865
--webserver_interface=127.20.113.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.60.keytab
--principal=kudu/127.20.113.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:39499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.113.62:41515,127.20.113.61:44499,127.20.113.60:34865
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 09:31:51.937573 8234 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.113.60" port: 34865 } attempt: 1
I20260806 09:31:51.938762 8238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:31:51.938264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.60:34865 (local address 127.0.0.1:60546)
0806 09:31:51.938489 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0806 09:31:51.938607 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.60:34865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260806 09:31:51.940063 8238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:31:51.939545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.60:34865 (local address 127.0.0.1:60562)
0806 09:31:51.939767 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0806 09:31:51.939892 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.60:34865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
W20260806 09:31:51.940371 8234 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.113.60:34865: Network error: Client connection negotiation failed: client connection to 127.20.113.60:34865: connect: Connection refused (error 111)
I20260806 09:31:51.999508 8137 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.113.61" port: 44499 } attempt: 5
Aug 06 09:31:52 dist-test-slave-45dx krb5kdc[8023](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008711, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.62@KRBTEST.COM for kudu/127.20.113.61@KRBTEST.COM
I20260806 09:31:52.008615 8234 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.113.60" port: 34865 } attempt: 2
I20260806 09:31:52.009644 8238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:31:52.009184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.60:34865 (local address 127.0.0.1:60578)
0806 09:31:52.009421 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0806 09:31:52.009494 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.60:34865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147}
I20260806 09:31:52.010803 8238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:31:52.010361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.60:34865 (local address 127.0.0.1:60590)
0806 09:31:52.010552 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0806 09:31:52.010647 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.60:34865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260806 09:31:52.011070 8234 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.113.60:34865: Network error: Client connection negotiation failed: client connection to 127.20.113.60:34865: connect: Connection refused (error 111)
I20260806 09:31:52.014890 8141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:31:52.000402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.61:44499 (local address 127.0.0.1:44294)
0806 09:31:52.000632 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0806 09:31:52.000655 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 09:31:52.000805 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:31:52.001194 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:31:52.001199 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:31:52.001612 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:31:52.001811 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:31:52.001819 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:31:52.002825 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:31:52.002833 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:31:52.003965 (+ 1132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:31:52.003979 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:31:52.004887 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:31:52.004893 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:31:52.005093 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:31:52.005745 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:31:52.005769 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:31:52.007922 (+ 2153us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:31:52.012665 (+ 4743us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:31:52.012669 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:31:52.012672 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:31:52.013052 (+ 380us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:31:52.013462 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:31:52.013465 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:31:52.013467 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:31:52.013594 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:31:52.014186 (+ 592us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:31:52.014190 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:31:52.014314 (+ 124us) client_negotiation.cc:769] Sending connection context
0806 09:31:52.014454 (+ 140us) client_negotiation.cc:240] Negotiation successful
0806 09:31:52.014667 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":82}
I20260806 09:31:52.015079 8243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:31:52.000475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44294 (local address 127.20.113.61:44499)
0806 09:31:52.000664 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0806 09:31:52.000672 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 09:31:52.000719 (+ 47us) server_negotiation.cc:407] Connection header received
0806 09:31:52.000957 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:31:52.000969 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:31:52.001050 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:31:52.001159 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:31:52.001994 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:31:52.002684 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:31:52.004150 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:31:52.004728 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:31:52.004864 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:31:52.008143 (+ 3279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:31:52.008149 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:31:52.008154 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:31:52.008195 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:31:52.012507 (+ 4312us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:31:52.013177 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:31:52.013182 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:31:52.013186 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:31:52.013352 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:31:52.013702 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:31:52.013706 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:31:52.013710 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:31:52.014068 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:31:52.014224 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0806 09:31:52.014571 (+ 347us) server_negotiation.cc:299] Negotiation successful
0806 09:31:52.014793 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260806 09:31:52.016012 8137 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.113.60" port: 34865 } has no permanent_uuid. Determining permanent_uuid...
I20260806 09:31:52.017158 8141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:31:52.016810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.60:34865 (local address 127.0.0.1:60604)
0806 09:31:52.016963 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0806 09:31:52.017033 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.60:34865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260806 09:31:52.018213 8141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:31:52.017790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.60:34865 (local address 127.0.0.1:60610)
0806 09:31:52.017964 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0806 09:31:52.018072 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.60:34865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260806 09:31:52.018431 8137 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.113.60:34865: Network error: Client connection negotiation failed: client connection to 127.20.113.60:34865: connect: Connection refused (error 111)
I20260806 09:31:52.069659 8137 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.113.60" port: 34865 } attempt: 1
I20260806 09:31:52.070860 8141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:31:52.070379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.60:34865 (local address 127.0.0.1:60616)
0806 09:31:52.070588 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0806 09:31:52.070682 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.60:34865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260806 09:31:52.071888 8141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:31:52.071559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.60:34865 (local address 127.0.0.1:60630)
0806 09:31:52.071708 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0806 09:31:52.071788 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.60:34865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260806 09:31:52.072103 8137 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.113.60:34865: Network error: Client connection negotiation failed: client connection to 127.20.113.60:34865: connect: Connection refused (error 111)
I20260806 09:31:52.102301 8234 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.113.60" port: 34865 } attempt: 3
I20260806 09:31:52.103600 8238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:31:52.103008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.60:34865 (local address 127.0.0.1:60634)
0806 09:31:52.103263 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0806 09:31:52.103396 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.60:34865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260806 09:31:52.104868 8238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:31:52.104352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.60:34865 (local address 127.0.0.1:60640)
0806 09:31:52.104622 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0806 09:31:52.104709 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.60:34865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150}
W20260806 09:31:52.105106 8234 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.113.60:34865: Network error: Client connection negotiation failed: client connection to 127.20.113.60:34865: connect: Connection refused (error 111)
I20260806 09:31:52.140331 8137 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.113.60" port: 34865 } attempt: 2
I20260806 09:31:52.141647 8141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:31:52.141145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.60:34865 (local address 127.0.0.1:60648)
0806 09:31:52.141374 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0806 09:31:52.141476 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.60:34865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260806 09:31:52.142674 8141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:31:52.142342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.60:34865 (local address 127.0.0.1:60654)
0806 09:31:52.142486 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0806 09:31:52.142572 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.60:34865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260806 09:31:52.142903 8137 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.113.60:34865: Network error: Client connection negotiation failed: client connection to 127.20.113.60:34865: connect: Connection refused (error 111)
I20260806 09:31:52.248391 8234 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.113.60" port: 34865 } attempt: 4
I20260806 09:31:52.249126 8137 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.113.60" port: 34865 } attempt: 3
I20260806 09:31:52.249755 8238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:31:52.249115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.60:34865 (local address 127.0.0.1:56334)
0806 09:31:52.249438 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0806 09:31:52.249553 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.60:34865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123}
I20260806 09:31:52.250192 8141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:31:52.249713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.60:34865 (local address 127.0.0.1:56344)
0806 09:31:52.249900 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0806 09:31:52.250004 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.60:34865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260806 09:31:52.250962 8238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:31:52.250595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.60:34865 (local address 127.0.0.1:56352)
0806 09:31:52.250774 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0806 09:31:52.250848 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.60:34865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260806 09:31:52.251164 8234 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.113.60:34865: Network error: Client connection negotiation failed: client connection to 127.20.113.60:34865: connect: Connection refused (error 111)
I20260806 09:31:52.251729 8141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:31:52.251316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.60:34865 (local address 127.0.0.1:56354)
0806 09:31:52.251548 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0806 09:31:52.251631 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.60:34865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
W20260806 09:31:52.251924 8137 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.113.60:34865: Network error: Client connection negotiation failed: client connection to 127.20.113.60:34865: connect: Connection refused (error 111)
W20260806 09:31:52.268288 8246 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:31:52.268757 8246 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:31:52.268922 8246 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:31:52.277563 8246 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 09:31:52.277714 8246 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:31:52.277846 8246 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:31:52.277957 8246 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 09:31:52.278048 8246 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 09:31:52.288672 8246 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:39499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786008495763054-20932-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786008495763054-20932-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.20.113.62:41515,127.20.113.61:44499,127.20.113.60:34865
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.60.keytab
--principal=kudu/127.20.113.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.113.60:34865
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786008495763054-20932-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.20.113.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786008495763054-20932-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:31:52.290231 8246 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:31:52.291617 8246 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:31:52.305513 8255 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:31:52.305619 8257 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:31:52.306241 8246 server_base.cc:1034] running on GCE node
W20260806 09:31:52.306268 8252 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:31:52.307153 8246 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:31:52.308717 8246 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:31:52.309986 8246 hybrid_clock.cc:648] HybridClock initialized: now 1786008712309941 us; error 48 us; skew 500 ppm
I20260806 09:31:52.310381 8246 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:31:52 dist-test-slave-45dx krb5kdc[8023](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008712, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:31:52.313534 8246 init.cc:377] Logged in from keytab as kudu/127.20.113.60@KRBTEST.COM (short username kudu)
I20260806 09:31:52.315081 8246 webserver.cc:460] Webserver started at http://127.20.113.60:40053/ using document root <none> and password file <none>
I20260806 09:31:52.315768 8246 fs_manager.cc:359] Metadata directory not provided
I20260806 09:31:52.315917 8246 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:31:52.316234 8246 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:31:52.319316 8246 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786008495763054-20932-0/minicluster-data/master-2/data/instance:
uuid: "a526ae086af444469aa58a0cb06523be"
format_stamp: "Formatted at 2026-08-06 09:31:52 on dist-test-slave-45dx"
server_key: "16b94ef2155fa4a8efd99f9b8c1af6c9"
server_key_iv: "cbc18975169db5ef45d6a1ec4734c944"
server_key_version: "encryptionkey@0"
I20260806 09:31:52.320132 8246 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786008495763054-20932-0/minicluster-data/master-2/wal/instance:
uuid: "a526ae086af444469aa58a0cb06523be"
format_stamp: "Formatted at 2026-08-06 09:31:52 on dist-test-slave-45dx"
server_key: "16b94ef2155fa4a8efd99f9b8c1af6c9"
server_key_iv: "cbc18975169db5ef45d6a1ec4734c944"
server_key_version: "encryptionkey@0"
I20260806 09:31:52.325233 8246 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 09:31:52.328773 8272 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:31:52.330086 8246 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260806 09:31:52.330298 8246 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786008495763054-20932-0/minicluster-data/master-2/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786008495763054-20932-0/minicluster-data/master-2/wal
uuid: "a526ae086af444469aa58a0cb06523be"
format_stamp: "Formatted at 2026-08-06 09:31:52 on dist-test-slave-45dx"
server_key: "16b94ef2155fa4a8efd99f9b8c1af6c9"
server_key_iv: "cbc18975169db5ef45d6a1ec4734c944"
server_key_version: "encryptionkey@0"
I20260806 09:31:52.330513 8246 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786008495763054-20932-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786008495763054-20932-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786008495763054-20932-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:31:52.361035 8246 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:31:52.364897 8246 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:31:52.365190 8246 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:31:52.384737 8246 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.60:34865
I20260806 09:31:52.384758 8349 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.60:34865 every 8 connection(s)
I20260806 09:31:52.386229 8246 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786008495763054-20932-0/minicluster-data/master-2/data/info.pb
I20260806 09:31:52.390309 8350 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:31:52.394637 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 8246
I20260806 09:31:52.394850 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786008495763054-20932-0/minicluster-data/master-2/wal/instance
I20260806 09:31:52.395328 20932 external_mini_cluster.cc:1428] Setting key 3c9364d83f758e82c5f3b5b1a630dce3
I20260806 09:31:52.397449 8350 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.113.62" port: 41515 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 09:31:52 dist-test-slave-45dx krb5kdc[8023](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008712, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.60@KRBTEST.COM for kudu/127.20.113.62@KRBTEST.COM
Aug 06 09:31:52 dist-test-slave-45dx krb5kdc[8023](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008710, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.113.60@KRBTEST.COM
I20260806 09:31:52.418722 8357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:31:52.399998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57074 (local address 127.20.113.62:41515)
0806 09:31:52.401299 (+ 1301us) server_negotiation.cc:206] Beginning negotiation
0806 09:31:52.401306 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:31:52.401325 (+ 19us) server_negotiation.cc:407] Connection header received
0806 09:31:52.401386 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:31:52.401391 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:31:52.401483 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:31:52.401624 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:31:52.403344 (+ 1720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:31:52.404528 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:31:52.406685 (+ 2157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:31:52.407331 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:31:52.407475 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:31:52.411890 (+ 4415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:31:52.411895 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:31:52.411899 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:31:52.411932 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:31:52.414964 (+ 3032us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:31:52.416044 (+ 1080us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:31:52.416049 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:31:52.416051 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:31:52.416168 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:31:52.416697 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:31:52.416703 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:31:52.416707 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:31:52.417107 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:31:52.417251 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0806 09:31:52.418148 (+ 897us) server_negotiation.cc:299] Negotiation successful
0806 09:31:52.418301 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1169,"thread_start_us":1023,"threads_started":1}
I20260806 09:31:52.420145 8356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:31:52.399928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:41515 (local address 127.0.0.1:57074)
0806 09:31:52.400757 (+ 829us) negotiation.cc:107] Waiting for socket to connect
0806 09:31:52.400827 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0806 09:31:52.401022 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:31:52.401723 (+ 701us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:31:52.401730 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:31:52.402570 (+ 840us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:31:52.403171 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:31:52.403188 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:31:52.404684 (+ 1496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:31:52.404688 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:31:52.406499 (+ 1811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:31:52.406515 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:31:52.407555 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:31:52.407560 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:31:52.407746 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:31:52.408598 (+ 852us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:31:52.408630 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:31:52.411699 (+ 3069us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:31:52.415105 (+ 3406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:31:52.415123 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:31:52.415135 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:31:52.415917 (+ 782us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:31:52.416294 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:31:52.416302 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:31:52.416308 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:31:52.416576 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:31:52.417235 (+ 659us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:31:52.417243 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:31:52.417682 (+ 439us) client_negotiation.cc:769] Sending connection context
0806 09:31:52.418056 (+ 374us) client_negotiation.cc:240] Negotiation successful
0806 09:31:52.418593 (+ 537us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":509,"thread_start_us":305,"threads_started":1}
I20260806 09:31:52.420105 8355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:31:52.397372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56358 (local address 127.20.113.60:34865)
0806 09:31:52.398265 (+ 893us) server_negotiation.cc:206] Beginning negotiation
0806 09:31:52.398282 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 09:31:52.398334 (+ 52us) server_negotiation.cc:407] Connection header received
0806 09:31:52.400543 (+ 2209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:31:52.400568 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:31:52.401024 (+ 456us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:31:52.401302 (+ 278us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:31:52.403142 (+ 1840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:31:52.404857 (+ 1715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:31:52.406709 (+ 1852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:31:52.407568 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:31:52.407735 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:31:52.412758 (+ 5023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:31:52.412767 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:31:52.412778 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:31:52.412810 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:31:52.416242 (+ 3432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:31:52.417196 (+ 954us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:31:52.417206 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:31:52.417210 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:31:52.417396 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:31:52.417782 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:31:52.417788 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:31:52.417791 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:31:52.418287 (+ 496us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:31:52.418612 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0806 09:31:52.418959 (+ 347us) server_negotiation.cc:299] Negotiation successful
0806 09:31:52.419232 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":649,"spinlock_wait_cycles":12544,"thread_start_us":423,"threads_started":1}
I20260806 09:31:52.422205 8350 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.113.61" port: 44499 } has no permanent_uuid. Determining permanent_uuid...
I20260806 09:31:52.429178 8137 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.113.60" port: 34865 } attempt: 4
Aug 06 09:31:52 dist-test-slave-45dx krb5kdc[8023](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008712, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.60@KRBTEST.COM for kudu/127.20.113.61@KRBTEST.COM
Aug 06 09:31:52 dist-test-slave-45dx krb5kdc[8023](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008710, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.113.62@KRBTEST.COM
Aug 06 09:31:52 dist-test-slave-45dx krb5kdc[8023](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008711, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.62@KRBTEST.COM for kudu/127.20.113.60@KRBTEST.COM
I20260806 09:31:52.439965 8356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:31:52.423073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.61:44499 (local address 127.0.0.1:56376)
0806 09:31:52.423254 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0806 09:31:52.423285 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0806 09:31:52.423386 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:31:52.424021 (+ 635us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:31:52.424025 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:31:52.424410 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:31:52.424564 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:31:52.424572 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:31:52.426162 (+ 1590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:31:52.426172 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:31:52.427492 (+ 1320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:31:52.427504 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:31:52.428697 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:31:52.428701 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:31:52.428855 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:31:52.429551 (+ 696us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:31:52.429579 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:31:52.432426 (+ 2847us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:31:52.436924 (+ 4498us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:31:52.436931 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:31:52.436936 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:31:52.437503 (+ 567us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:31:52.438044 (+ 541us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:31:52.438049 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:31:52.438053 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:31:52.438257 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:31:52.438842 (+ 585us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:31:52.438848 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:31:52.439029 (+ 181us) client_negotiation.cc:769] Sending connection context
0806 09:31:52.439201 (+ 172us) client_negotiation.cc:240] Negotiation successful
0806 09:31:52.439512 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68}
I20260806 09:31:52.439991 8243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:31:52.423164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56376 (local address 127.20.113.61:44499)
0806 09:31:52.423365 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0806 09:31:52.423371 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:31:52.423391 (+ 20us) server_negotiation.cc:407] Connection header received
0806 09:31:52.423494 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:31:52.423499 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:31:52.423599 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:31:52.423700 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:31:52.424741 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:31:52.425993 (+ 1252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:31:52.427645 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:31:52.428427 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:31:52.428572 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:31:52.432669 (+ 4097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:31:52.432676 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:31:52.432687 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:31:52.432738 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:31:52.436729 (+ 3991us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:31:52.437667 (+ 938us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:31:52.437673 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:31:52.437677 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:31:52.437877 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:31:52.438424 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:31:52.438428 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:31:52.438431 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:31:52.438702 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:31:52.438887 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0806 09:31:52.439335 (+ 448us) server_negotiation.cc:299] Negotiation successful
0806 09:31:52.439560 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 09:31:52.441915 8350 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.113.60" port: 34865 } has no permanent_uuid. Determining permanent_uuid...
I20260806 09:31:52.442322 8357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:31:52.423688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57080 (local address 127.20.113.62:41515)
0806 09:31:52.423926 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0806 09:31:52.423929 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 09:31:52.423943 (+ 14us) server_negotiation.cc:407] Connection header received
0806 09:31:52.424109 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:31:52.424117 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:31:52.424193 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:31:52.424304 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:31:52.426017 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:31:52.427132 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:31:52.429044 (+ 1912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:31:52.430074 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:31:52.430194 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:31:52.435553 (+ 5359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:31:52.435560 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:31:52.435564 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:31:52.435596 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:31:52.438752 (+ 3156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:31:52.439597 (+ 845us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:31:52.439602 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:31:52.439606 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:31:52.439744 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:31:52.440218 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:31:52.440223 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:31:52.440226 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:31:52.440680 (+ 454us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:31:52.440977 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0806 09:31:52.441507 (+ 530us) server_negotiation.cc:299] Negotiation successful
0806 09:31:52.441724 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152}
I20260806 09:31:52.447455 8141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:31:52.429740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.60:34865 (local address 127.0.0.1:56372)
0806 09:31:52.429918 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0806 09:31:52.429940 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 09:31:52.430053 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:31:52.430408 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:31:52.430414 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:31:52.430813 (+ 399us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:31:52.431058 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:31:52.431074 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:31:52.432209 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:31:52.432216 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:31:52.434290 (+ 2074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:31:52.434307 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:31:52.435440 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:31:52.435448 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:31:52.435662 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:31:52.436342 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:31:52.436372 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:31:52.439650 (+ 3278us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:31:52.444261 (+ 4611us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:31:52.444266 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:31:52.444269 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:31:52.444834 (+ 565us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:31:52.445312 (+ 478us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:31:52.445318 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:31:52.445324 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:31:52.445533 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:31:52.446387 (+ 854us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:31:52.446390 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:31:52.446580 (+ 190us) client_negotiation.cc:769] Sending connection context
0806 09:31:52.446741 (+ 161us) client_negotiation.cc:240] Negotiation successful
0806 09:31:52.447040 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260806 09:31:52.447500 8355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:31:52.429837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56372 (local address 127.20.113.60:34865)
0806 09:31:52.430043 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0806 09:31:52.430049 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:31:52.430072 (+ 23us) server_negotiation.cc:407] Connection header received
0806 09:31:52.430173 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:31:52.430178 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:31:52.430254 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:31:52.430362 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:31:52.431251 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:31:52.432062 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:31:52.434446 (+ 2384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:31:52.435270 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:31:52.435408 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:31:52.439896 (+ 4488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:31:52.439902 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:31:52.439907 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:31:52.439954 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:31:52.444093 (+ 4139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:31:52.445017 (+ 924us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:31:52.445023 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:31:52.445027 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:31:52.445181 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:31:52.445695 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:31:52.445700 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:31:52.445703 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:31:52.446137 (+ 434us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:31:52.446292 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0806 09:31:52.446886 (+ 594us) server_negotiation.cc:299] Negotiation successful
0806 09:31:52.447171 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260806 09:31:52.453147 8137 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 51a55182913d414b8cb7cf5eef9e1f3d: Bootstrap starting.
Aug 06 09:31:52 dist-test-slave-45dx krb5kdc[8023](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008712, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.60@KRBTEST.COM for kudu/127.20.113.60@KRBTEST.COM
Aug 06 09:31:52 dist-test-slave-45dx krb5kdc[8023](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008710, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.113.61@KRBTEST.COM
I20260806 09:31:52.457532 8137 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 51a55182913d414b8cb7cf5eef9e1f3d: Neither blocks nor log segments found. Creating new log.
I20260806 09:31:52.458890 8137 log.cc:824] T 00000000000000000000000000000000 P 51a55182913d414b8cb7cf5eef9e1f3d: Log is configured to *not* fsync() on all Append() calls
I20260806 09:31:52.461715 8356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:31:52.442927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.60:34865 (local address 127.0.0.1:56388)
0806 09:31:52.443338 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0806 09:31:52.443358 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 09:31:52.443465 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:31:52.446081 (+ 2616us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:31:52.446090 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:31:52.446588 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:31:52.446745 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:31:52.446757 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:31:52.448667 (+ 1910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:31:52.448674 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:31:52.450180 (+ 1506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:31:52.450197 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:31:52.451190 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:31:52.451195 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:31:52.451442 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:31:52.452075 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:31:52.452104 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:31:52.454843 (+ 2739us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:31:52.458874 (+ 4031us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:31:52.458880 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:31:52.458884 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:31:52.459339 (+ 455us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:31:52.459712 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:31:52.459718 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:31:52.459722 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:31:52.459900 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:31:52.460629 (+ 729us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:31:52.460633 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:31:52.460783 (+ 150us) client_negotiation.cc:769] Sending connection context
0806 09:31:52.460985 (+ 202us) client_negotiation.cc:240] Negotiation successful
0806 09:31:52.461336 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"mutex_wait_us":3}
I20260806 09:31:52.462363 8243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:31:52.444485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56382 (local address 127.20.113.61:44499)
0806 09:31:52.444665 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0806 09:31:52.444674 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 09:31:52.444693 (+ 19us) server_negotiation.cc:407] Connection header received
0806 09:31:52.444748 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:31:52.444754 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:31:52.444828 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:31:52.444953 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:31:52.446527 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:31:52.447708 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:31:52.449806 (+ 2098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:31:52.450485 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:31:52.450631 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:31:52.455860 (+ 5229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:31:52.455866 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:31:52.455871 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:31:52.455912 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:31:52.458875 (+ 2963us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:31:52.459777 (+ 902us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:31:52.459782 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:31:52.459788 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:31:52.459942 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:31:52.460502 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:31:52.460507 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:31:52.460510 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:31:52.460800 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:31:52.461123 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0806 09:31:52.461472 (+ 349us) server_negotiation.cc:299] Negotiation successful
0806 09:31:52.461708 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 09:31:52.462741 8358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:31:52.442750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56388 (local address 127.20.113.60:34865)
0806 09:31:52.445752 (+ 3002us) server_negotiation.cc:206] Beginning negotiation
0806 09:31:52.445757 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:31:52.445787 (+ 30us) server_negotiation.cc:407] Connection header received
0806 09:31:52.445837 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:31:52.445842 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:31:52.445908 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:31:52.446023 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:31:52.446918 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:31:52.448518 (+ 1600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:31:52.450346 (+ 1828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:31:52.451016 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:31:52.451422 (+ 406us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:31:52.455103 (+ 3681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:31:52.455112 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:31:52.455118 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:31:52.455159 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:31:52.458722 (+ 3563us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:31:52.459456 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:31:52.459464 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:31:52.459467 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:31:52.459596 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:31:52.460030 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:31:52.460038 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:31:52.460043 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:31:52.460424 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:31:52.460569 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0806 09:31:52.462120 (+ 1551us) server_negotiation.cc:299] Negotiation successful
0806 09:31:52.462324 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2862,"thread_start_us":2691,"threads_started":1}
I20260806 09:31:52.463131 8137 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 51a55182913d414b8cb7cf5eef9e1f3d: No bootstrap required, opened a new log
I20260806 09:31:52.467175 20932 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 09:31:52.468343 8350 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a526ae086af444469aa58a0cb06523be: Bootstrap starting.
I20260806 09:31:52.472131 8350 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a526ae086af444469aa58a0cb06523be: Neither blocks nor log segments found. Creating new log.
I20260806 09:31:52.473538 8350 log.cc:824] T 00000000000000000000000000000000 P a526ae086af444469aa58a0cb06523be: Log is configured to *not* fsync() on all Append() calls
I20260806 09:31:52.474450 8137 raft_consensus.cc:348] T 00000000000000000000000000000000 P 51a55182913d414b8cb7cf5eef9e1f3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51a55182913d414b8cb7cf5eef9e1f3d" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 41515 } } peers { permanent_uuid: "91ace7d8479a4eb8abca1d23fa091e5f" member_type: VOTER last_known_addr { host: "127.20.113.61" port: 44499 } } peers { permanent_uuid: "a526ae086af444469aa58a0cb06523be" member_type: VOTER last_known_addr { host: "127.20.113.60" port: 34865 } }
I20260806 09:31:52.474937 8137 raft_consensus.cc:374] T 00000000000000000000000000000000 P 51a55182913d414b8cb7cf5eef9e1f3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:31:52.475077 8137 raft_consensus.cc:729] T 00000000000000000000000000000000 P 51a55182913d414b8cb7cf5eef9e1f3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51a55182913d414b8cb7cf5eef9e1f3d, State: Initialized, Role: FOLLOWER
I20260806 09:31:52.475953 8137 consensus_queue.cc:260] T 00000000000000000000000000000000 P 51a55182913d414b8cb7cf5eef9e1f3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51a55182913d414b8cb7cf5eef9e1f3d" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 41515 } } peers { permanent_uuid: "91ace7d8479a4eb8abca1d23fa091e5f" member_type: VOTER last_known_addr { host: "127.20.113.61" port: 44499 } } peers { permanent_uuid: "a526ae086af444469aa58a0cb06523be" member_type: VOTER last_known_addr { host: "127.20.113.60" port: 34865 } }
I20260806 09:31:52.483155 8368 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51a55182913d414b8cb7cf5eef9e1f3d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51a55182913d414b8cb7cf5eef9e1f3d" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 41515 } } peers { permanent_uuid: "91ace7d8479a4eb8abca1d23fa091e5f" member_type: VOTER last_known_addr { host: "127.20.113.61" port: 44499 } } peers { permanent_uuid: "a526ae086af444469aa58a0cb06523be" member_type: VOTER last_known_addr { host: "127.20.113.60" port: 34865 } } }
I20260806 09:31:52.483458 8350 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a526ae086af444469aa58a0cb06523be: No bootstrap required, opened a new log
I20260806 09:31:52.483554 8368 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51a55182913d414b8cb7cf5eef9e1f3d [sys.catalog]: This master's current role is: FOLLOWER
I20260806 09:31:52.485383 8137 sys_catalog.cc:564] T 00000000000000000000000000000000 P 51a55182913d414b8cb7cf5eef9e1f3d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 09:31:52.494467 8243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:31:52.472468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56390 (local address 127.20.113.61:44499)
0806 09:31:52.473517 (+ 1049us) server_negotiation.cc:206] Beginning negotiation
0806 09:31:52.473523 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:31:52.473538 (+ 15us) server_negotiation.cc:407] Connection header received
0806 09:31:52.473613 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:31:52.473619 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:31:52.473697 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:31:52.473791 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:31:52.477087 (+ 3296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:31:52.478574 (+ 1487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:31:52.481185 (+ 2611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:31:52.482100 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:31:52.482222 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:31:52.486422 (+ 4200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:31:52.486428 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:31:52.486432 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:31:52.486475 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:31:52.491098 (+ 4623us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:31:52.491924 (+ 826us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:31:52.491930 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:31:52.491933 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:31:52.492112 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:31:52.492620 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:31:52.492627 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:31:52.492630 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:31:52.493007 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:31:52.493245 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0806 09:31:52.493858 (+ 613us) server_negotiation.cc:299] Negotiation successful
0806 09:31:52.494114 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":924}
I20260806 09:31:52.505308 8358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:31:52.474101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56404 (local address 127.20.113.60:34865)
0806 09:31:52.474312 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0806 09:31:52.474317 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:31:52.488773 (+ 14456us) server_negotiation.cc:407] Connection header received
0806 09:31:52.488882 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:31:52.488891 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:31:52.489024 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:31:52.489140 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:31:52.490831 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:31:52.491924 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:31:52.494230 (+ 2306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:31:52.494865 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:31:52.494967 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:31:52.499525 (+ 4558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:31:52.499535 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:31:52.499540 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:31:52.499587 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:31:52.502423 (+ 2836us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:31:52.503148 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:31:52.503152 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:31:52.503154 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:31:52.503278 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:31:52.503706 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:31:52.503710 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:31:52.503712 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:31:52.504007 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:31:52.504279 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0806 09:31:52.504693 (+ 414us) server_negotiation.cc:299] Negotiation successful
0806 09:31:52.504951 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 09:31:52.505712 8350 raft_consensus.cc:348] T 00000000000000000000000000000000 P a526ae086af444469aa58a0cb06523be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51a55182913d414b8cb7cf5eef9e1f3d" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 41515 } } peers { permanent_uuid: "91ace7d8479a4eb8abca1d23fa091e5f" member_type: VOTER last_known_addr { host: "127.20.113.61" port: 44499 } } peers { permanent_uuid: "a526ae086af444469aa58a0cb06523be" member_type: VOTER last_known_addr { host: "127.20.113.60" port: 34865 } }
I20260806 09:31:52.506229 8350 raft_consensus.cc:374] T 00000000000000000000000000000000 P a526ae086af444469aa58a0cb06523be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:31:52.506392 8350 raft_consensus.cc:729] T 00000000000000000000000000000000 P a526ae086af444469aa58a0cb06523be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a526ae086af444469aa58a0cb06523be, State: Initialized, Role: FOLLOWER
I20260806 09:31:52.507228 8350 consensus_queue.cc:260] T 00000000000000000000000000000000 P a526ae086af444469aa58a0cb06523be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51a55182913d414b8cb7cf5eef9e1f3d" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 41515 } } peers { permanent_uuid: "91ace7d8479a4eb8abca1d23fa091e5f" member_type: VOTER last_known_addr { host: "127.20.113.61" port: 44499 } } peers { permanent_uuid: "a526ae086af444469aa58a0cb06523be" member_type: VOTER last_known_addr { host: "127.20.113.60" port: 34865 } }
I20260806 09:31:52.508515 8376 sys_catalog.cc:455] T 00000000000000000000000000000000 P a526ae086af444469aa58a0cb06523be [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51a55182913d414b8cb7cf5eef9e1f3d" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 41515 } } peers { permanent_uuid: "91ace7d8479a4eb8abca1d23fa091e5f" member_type: VOTER last_known_addr { host: "127.20.113.61" port: 44499 } } peers { permanent_uuid: "a526ae086af444469aa58a0cb06523be" member_type: VOTER last_known_addr { host: "127.20.113.60" port: 34865 } } }
I20260806 09:31:52.509073 8376 sys_catalog.cc:458] T 00000000000000000000000000000000 P a526ae086af444469aa58a0cb06523be [sys.catalog]: This master's current role is: FOLLOWER
I20260806 09:31:52.510233 8350 sys_catalog.cc:564] T 00000000000000000000000000000000 P a526ae086af444469aa58a0cb06523be [sys.catalog]: configured and running, proceeding with master startup.
I20260806 09:31:52.516256 8357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:31:52.480042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57082 (local address 127.20.113.62:41515)
0806 09:31:52.480217 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0806 09:31:52.480224 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:31:52.493014 (+ 12790us) server_negotiation.cc:407] Connection header received
0806 09:31:52.493315 (+ 301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:31:52.493323 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:31:52.493433 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:31:52.493515 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:31:52.495829 (+ 2314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:31:52.497351 (+ 1522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:31:52.504439 (+ 7088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:31:52.505121 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:31:52.505216 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:31:52.510280 (+ 5064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:31:52.510291 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:31:52.510294 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:31:52.510378 (+ 84us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:31:52.513390 (+ 3012us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:31:52.514118 (+ 728us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:31:52.514123 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:31:52.514127 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:31:52.514286 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:31:52.514675 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:31:52.514681 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:31:52.514684 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:31:52.514983 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:31:52.515296 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0806 09:31:52.515613 (+ 317us) server_negotiation.cc:299] Negotiation successful
0806 09:31:52.515845 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
W20260806 09:31:52.529536 8396 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a526ae086af444469aa58a0cb06523be: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 09:31:52.529760 8396 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260806 09:31:52.532241 8070 tablet.cc:2369] T 00000000000000000000000000000000 P 51a55182913d414b8cb7cf5eef9e1f3d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 09:31:52.525104 8246 master_runner.cc:186] Error getting master registration for 127.20.113.62:41515: OK, instance_id { permanent_uuid: "51a55182913d414b8cb7cf5eef9e1f3d" instance_seqno: 1786008711292920 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 09:31:52.540511 8246 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260806 09:31:52.547263 8402 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 51a55182913d414b8cb7cf5eef9e1f3d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 09:31:52.547515 8402 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 09:31:52.550386 8234 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.113.60" port: 34865 } attempt: 5
I20260806 09:31:52.543448 8039 master_runner.cc:186] Error getting master registration for 127.20.113.61:44499: OK, instance_id { permanent_uuid: "91ace7d8479a4eb8abca1d23fa091e5f" instance_seqno: 1786008711813345 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 09:31:52.554163 8039 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Aug 06 09:31:52 dist-test-slave-45dx krb5kdc[8023](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008711, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.61@KRBTEST.COM for kudu/127.20.113.60@KRBTEST.COM
I20260806 09:31:52.565554 8238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:31:52.550953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.60:34865 (local address 127.0.0.1:56416)
0806 09:31:52.551236 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0806 09:31:52.551255 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 09:31:52.551369 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:31:52.551935 (+ 566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:31:52.551941 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:31:52.552498 (+ 557us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:31:52.552708 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:31:52.552721 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:31:52.554308 (+ 1587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:31:52.554315 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:31:52.555347 (+ 1032us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:31:52.555359 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:31:52.556148 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:31:52.556170 (+ 22us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:31:52.556429 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:31:52.557058 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:31:52.557075 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:31:52.559131 (+ 2056us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:31:52.562258 (+ 3127us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:31:52.562263 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:31:52.562267 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:31:52.562664 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:31:52.563121 (+ 457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:31:52.563126 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:31:52.563130 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:31:52.563272 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:31:52.563783 (+ 511us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:31:52.563788 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:31:52.563944 (+ 156us) client_negotiation.cc:769] Sending connection context
0806 09:31:52.564911 (+ 967us) client_negotiation.cc:240] Negotiation successful
0806 09:31:52.565199 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143}
I20260806 09:31:52.566425 8358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:31:52.551064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56416 (local address 127.20.113.60:34865)
0806 09:31:52.551258 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0806 09:31:52.551264 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:31:52.551499 (+ 235us) server_negotiation.cc:407] Connection header received
0806 09:31:52.551565 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:31:52.551570 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:31:52.551648 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:31:52.551741 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:31:52.552879 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:31:52.554108 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:31:52.555494 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:31:52.555998 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:31:52.556137 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:31:52.559327 (+ 3190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:31:52.559331 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:31:52.559334 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:31:52.559362 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:31:52.562084 (+ 2722us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:31:52.562782 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:31:52.562788 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:31:52.562792 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:31:52.563004 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:31:52.563382 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:31:52.563388 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:31:52.563391 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:31:52.563663 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:31:52.563834 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0806 09:31:52.565861 (+ 2027us) server_negotiation.cc:299] Negotiation successful
0806 09:31:52.566099 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260806 09:31:52.571455 8234 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91ace7d8479a4eb8abca1d23fa091e5f: Bootstrap starting.
I20260806 09:31:52.575686 8234 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 91ace7d8479a4eb8abca1d23fa091e5f: Neither blocks nor log segments found. Creating new log.
I20260806 09:31:52.576797 8234 log.cc:824] T 00000000000000000000000000000000 P 91ace7d8479a4eb8abca1d23fa091e5f: Log is configured to *not* fsync() on all Append() calls
I20260806 09:31:52.579411 8234 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91ace7d8479a4eb8abca1d23fa091e5f: No bootstrap required, opened a new log
I20260806 09:31:52.589049 8234 raft_consensus.cc:348] T 00000000000000000000000000000000 P 91ace7d8479a4eb8abca1d23fa091e5f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51a55182913d414b8cb7cf5eef9e1f3d" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 41515 } } peers { permanent_uuid: "91ace7d8479a4eb8abca1d23fa091e5f" member_type: VOTER last_known_addr { host: "127.20.113.61" port: 44499 } } peers { permanent_uuid: "a526ae086af444469aa58a0cb06523be" member_type: VOTER last_known_addr { host: "127.20.113.60" port: 34865 } }
I20260806 09:31:52.589505 8234 raft_consensus.cc:374] T 00000000000000000000000000000000 P 91ace7d8479a4eb8abca1d23fa091e5f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:31:52.589627 8234 raft_consensus.cc:729] T 00000000000000000000000000000000 P 91ace7d8479a4eb8abca1d23fa091e5f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ace7d8479a4eb8abca1d23fa091e5f, State: Initialized, Role: FOLLOWER
I20260806 09:31:52.590340 8234 consensus_queue.cc:260] T 00000000000000000000000000000000 P 91ace7d8479a4eb8abca1d23fa091e5f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51a55182913d414b8cb7cf5eef9e1f3d" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 41515 } } peers { permanent_uuid: "91ace7d8479a4eb8abca1d23fa091e5f" member_type: VOTER last_known_addr { host: "127.20.113.61" port: 44499 } } peers { permanent_uuid: "a526ae086af444469aa58a0cb06523be" member_type: VOTER last_known_addr { host: "127.20.113.60" port: 34865 } }
I20260806 09:31:52.591360 8405 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91ace7d8479a4eb8abca1d23fa091e5f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51a55182913d414b8cb7cf5eef9e1f3d" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 41515 } } peers { permanent_uuid: "91ace7d8479a4eb8abca1d23fa091e5f" member_type: VOTER last_known_addr { host: "127.20.113.61" port: 44499 } } peers { permanent_uuid: "a526ae086af444469aa58a0cb06523be" member_type: VOTER last_known_addr { host: "127.20.113.60" port: 34865 } } }
I20260806 09:31:52.591846 8405 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91ace7d8479a4eb8abca1d23fa091e5f [sys.catalog]: This master's current role is: FOLLOWER
I20260806 09:31:52.592629 8234 sys_catalog.cc:564] T 00000000000000000000000000000000 P 91ace7d8479a4eb8abca1d23fa091e5f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 09:31:52.614202 8146 master_runner.cc:263] No leader master found from master 91ace7d8479a4eb8abca1d23fa091e5f
I20260806 09:31:52.614359 8146 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260806 09:31:52.617277 8280 tablet.cc:2369] T 00000000000000000000000000000000 P a526ae086af444469aa58a0cb06523be: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 09:31:52.619195 8422 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 91ace7d8479a4eb8abca1d23fa091e5f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 09:31:52.619328 8422 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 09:31:52.633503 8368 raft_consensus.cc:482] T 00000000000000000000000000000000 P 51a55182913d414b8cb7cf5eef9e1f3d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 09:31:52.633819 8368 raft_consensus.cc:504] T 00000000000000000000000000000000 P 51a55182913d414b8cb7cf5eef9e1f3d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51a55182913d414b8cb7cf5eef9e1f3d" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 41515 } } peers { permanent_uuid: "91ace7d8479a4eb8abca1d23fa091e5f" member_type: VOTER last_known_addr { host: "127.20.113.61" port: 44499 } } peers { permanent_uuid: "a526ae086af444469aa58a0cb06523be" member_type: VOTER last_known_addr { host: "127.20.113.60" port: 34865 } }
I20260806 09:31:52.635349 8368 leader_election.cc:288] T 00000000000000000000000000000000 P 51a55182913d414b8cb7cf5eef9e1f3d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91ace7d8479a4eb8abca1d23fa091e5f (127.20.113.61:44499), a526ae086af444469aa58a0cb06523be (127.20.113.60:34865)
I20260806 09:31:52.636650 8204 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "51a55182913d414b8cb7cf5eef9e1f3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ace7d8479a4eb8abca1d23fa091e5f" is_pre_election: true
I20260806 09:31:52.637262 8204 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 91ace7d8479a4eb8abca1d23fa091e5f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 51a55182913d414b8cb7cf5eef9e1f3d in term 0.
I20260806 09:31:52.638021 8063 leader_election.cc:302] T 00000000000000000000000000000000 P 51a55182913d414b8cb7cf5eef9e1f3d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 51a55182913d414b8cb7cf5eef9e1f3d, 91ace7d8479a4eb8abca1d23fa091e5f; no voters:
I20260806 09:31:52.639039 8309 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "51a55182913d414b8cb7cf5eef9e1f3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a526ae086af444469aa58a0cb06523be" is_pre_election: true
I20260806 09:31:52.639626 8309 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a526ae086af444469aa58a0cb06523be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 51a55182913d414b8cb7cf5eef9e1f3d in term 0.
I20260806 09:31:52.640682 8368 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 51a55182913d414b8cb7cf5eef9e1f3d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 09:31:52.640828 8368 raft_consensus.cc:482] T 00000000000000000000000000000000 P 51a55182913d414b8cb7cf5eef9e1f3d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 09:31:52.640925 8368 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 51a55182913d414b8cb7cf5eef9e1f3d [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:31:52.652545 8368 raft_consensus.cc:504] T 00000000000000000000000000000000 P 51a55182913d414b8cb7cf5eef9e1f3d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51a55182913d414b8cb7cf5eef9e1f3d" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 41515 } } peers { permanent_uuid: "91ace7d8479a4eb8abca1d23fa091e5f" member_type: VOTER last_known_addr { host: "127.20.113.61" port: 44499 } } peers { permanent_uuid: "a526ae086af444469aa58a0cb06523be" member_type: VOTER last_known_addr { host: "127.20.113.60" port: 34865 } }
I20260806 09:31:52.655599 8204 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "51a55182913d414b8cb7cf5eef9e1f3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ace7d8479a4eb8abca1d23fa091e5f"
I20260806 09:31:52.655835 8204 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 91ace7d8479a4eb8abca1d23fa091e5f [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:31:52.659412 8204 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 91ace7d8479a4eb8abca1d23fa091e5f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51a55182913d414b8cb7cf5eef9e1f3d in term 1.
I20260806 09:31:52.659984 8063 leader_election.cc:302] T 00000000000000000000000000000000 P 51a55182913d414b8cb7cf5eef9e1f3d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 51a55182913d414b8cb7cf5eef9e1f3d, 91ace7d8479a4eb8abca1d23fa091e5f; no voters:
I20260806 09:31:52.662982 8309 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "51a55182913d414b8cb7cf5eef9e1f3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a526ae086af444469aa58a0cb06523be"
I20260806 09:31:52.663224 8309 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a526ae086af444469aa58a0cb06523be [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:31:52.669358 8309 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a526ae086af444469aa58a0cb06523be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51a55182913d414b8cb7cf5eef9e1f3d in term 1.
I20260806 09:31:52.669934 8368 leader_election.cc:288] T 00000000000000000000000000000000 P 51a55182913d414b8cb7cf5eef9e1f3d [CANDIDATE]: Term 1 election: Requested vote from peers 91ace7d8479a4eb8abca1d23fa091e5f (127.20.113.61:44499), a526ae086af444469aa58a0cb06523be (127.20.113.60:34865)
I20260806 09:31:52.670207 8368 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 51a55182913d414b8cb7cf5eef9e1f3d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:31:52.670399 8368 raft_consensus.cc:686] T 00000000000000000000000000000000 P 51a55182913d414b8cb7cf5eef9e1f3d [term 1 LEADER]: Becoming Leader. State: Replica: 51a55182913d414b8cb7cf5eef9e1f3d, State: Running, Role: LEADER
I20260806 09:31:52.671059 8368 consensus_queue.cc:237] T 00000000000000000000000000000000 P 51a55182913d414b8cb7cf5eef9e1f3d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51a55182913d414b8cb7cf5eef9e1f3d" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 41515 } } peers { permanent_uuid: "91ace7d8479a4eb8abca1d23fa091e5f" member_type: VOTER last_known_addr { host: "127.20.113.61" port: 44499 } } peers { permanent_uuid: "a526ae086af444469aa58a0cb06523be" member_type: VOTER last_known_addr { host: "127.20.113.60" port: 34865 } }
I20260806 09:31:52.694198 8424 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51a55182913d414b8cb7cf5eef9e1f3d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 51a55182913d414b8cb7cf5eef9e1f3d. Latest consensus state: current_term: 1 leader_uuid: "51a55182913d414b8cb7cf5eef9e1f3d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51a55182913d414b8cb7cf5eef9e1f3d" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 41515 } } peers { permanent_uuid: "91ace7d8479a4eb8abca1d23fa091e5f" member_type: VOTER last_known_addr { host: "127.20.113.61" port: 44499 } } peers { permanent_uuid: "a526ae086af444469aa58a0cb06523be" member_type: VOTER last_known_addr { host: "127.20.113.60" port: 34865 } } }
I20260806 09:31:52.694695 8424 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51a55182913d414b8cb7cf5eef9e1f3d [sys.catalog]: This master's current role is: LEADER
I20260806 09:31:52.704190 8426 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 09:31:52.706607 8426 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 09:31:52.725742 8309 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a526ae086af444469aa58a0cb06523be [term 1 FOLLOWER]: Refusing update from remote peer 51a55182913d414b8cb7cf5eef9e1f3d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 09:31:52.726756 8424 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 51a55182913d414b8cb7cf5eef9e1f3d [LEADER]: Connected to new peer: Peer: permanent_uuid: "a526ae086af444469aa58a0cb06523be" member_type: VOTER last_known_addr { host: "127.20.113.60" port: 34865 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 09:31:52.736675 8204 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 91ace7d8479a4eb8abca1d23fa091e5f [term 1 FOLLOWER]: Refusing update from remote peer 51a55182913d414b8cb7cf5eef9e1f3d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 09:31:52.737591 8424 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 51a55182913d414b8cb7cf5eef9e1f3d [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ace7d8479a4eb8abca1d23fa091e5f" member_type: VOTER last_known_addr { host: "127.20.113.61" port: 44499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 09:31:52.762874 8376 sys_catalog.cc:455] T 00000000000000000000000000000000 P a526ae086af444469aa58a0cb06523be [sys.catalog]: SysCatalogTable state changed. Reason: New leader 51a55182913d414b8cb7cf5eef9e1f3d. Latest consensus state: current_term: 1 leader_uuid: "51a55182913d414b8cb7cf5eef9e1f3d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51a55182913d414b8cb7cf5eef9e1f3d" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 41515 } } peers { permanent_uuid: "91ace7d8479a4eb8abca1d23fa091e5f" member_type: VOTER last_known_addr { host: "127.20.113.61" port: 44499 } } peers { permanent_uuid: "a526ae086af444469aa58a0cb06523be" member_type: VOTER last_known_addr { host: "127.20.113.60" port: 34865 } } }
I20260806 09:31:52.763186 8376 sys_catalog.cc:458] T 00000000000000000000000000000000 P a526ae086af444469aa58a0cb06523be [sys.catalog]: This master's current role is: FOLLOWER
I20260806 09:31:52.764359 8405 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91ace7d8479a4eb8abca1d23fa091e5f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 51a55182913d414b8cb7cf5eef9e1f3d. Latest consensus state: current_term: 1 leader_uuid: "51a55182913d414b8cb7cf5eef9e1f3d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51a55182913d414b8cb7cf5eef9e1f3d" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 41515 } } peers { permanent_uuid: "91ace7d8479a4eb8abca1d23fa091e5f" member_type: VOTER last_known_addr { host: "127.20.113.61" port: 44499 } } peers { permanent_uuid: "a526ae086af444469aa58a0cb06523be" member_type: VOTER last_known_addr { host: "127.20.113.60" port: 34865 } } }
I20260806 09:31:52.764683 8405 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91ace7d8479a4eb8abca1d23fa091e5f [sys.catalog]: This master's current role is: FOLLOWER
I20260806 09:31:52.767326 8427 mvcc.cc:205] Tried to move back new op lower bound from 7315491687313448960 to 7315491687105773568. Current Snapshot: MvccSnapshot[applied={T|T < 7315491687313448960}]
I20260806 09:31:52.776844 8405 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91ace7d8479a4eb8abca1d23fa091e5f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "51a55182913d414b8cb7cf5eef9e1f3d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51a55182913d414b8cb7cf5eef9e1f3d" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 41515 } } peers { permanent_uuid: "91ace7d8479a4eb8abca1d23fa091e5f" member_type: VOTER last_known_addr { host: "127.20.113.61" port: 44499 } } peers { permanent_uuid: "a526ae086af444469aa58a0cb06523be" member_type: VOTER last_known_addr { host: "127.20.113.60" port: 34865 } } }
I20260806 09:31:52.777109 8405 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91ace7d8479a4eb8abca1d23fa091e5f [sys.catalog]: This master's current role is: FOLLOWER
I20260806 09:31:52.785725 8368 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51a55182913d414b8cb7cf5eef9e1f3d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "51a55182913d414b8cb7cf5eef9e1f3d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51a55182913d414b8cb7cf5eef9e1f3d" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 41515 } } peers { permanent_uuid: "91ace7d8479a4eb8abca1d23fa091e5f" member_type: VOTER last_known_addr { host: "127.20.113.61" port: 44499 } } peers { permanent_uuid: "a526ae086af444469aa58a0cb06523be" member_type: VOTER last_known_addr { host: "127.20.113.60" port: 34865 } } }
I20260806 09:31:52.786070 8368 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51a55182913d414b8cb7cf5eef9e1f3d [sys.catalog]: This master's current role is: LEADER
I20260806 09:31:52.786445 8368 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51a55182913d414b8cb7cf5eef9e1f3d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "51a55182913d414b8cb7cf5eef9e1f3d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51a55182913d414b8cb7cf5eef9e1f3d" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 41515 } } peers { permanent_uuid: "91ace7d8479a4eb8abca1d23fa091e5f" member_type: VOTER last_known_addr { host: "127.20.113.61" port: 44499 } } peers { permanent_uuid: "a526ae086af444469aa58a0cb06523be" member_type: VOTER last_known_addr { host: "127.20.113.60" port: 34865 } } }
I20260806 09:31:52.786742 8368 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51a55182913d414b8cb7cf5eef9e1f3d [sys.catalog]: This master's current role is: LEADER
I20260806 09:31:52.787544 8426 catalog_manager.cc:1347] Generated new cluster ID: f29cfc3e32b74cd084b529405e5ae619
I20260806 09:31:52.787640 8426 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 09:31:52.801254 8376 sys_catalog.cc:455] T 00000000000000000000000000000000 P a526ae086af444469aa58a0cb06523be [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "51a55182913d414b8cb7cf5eef9e1f3d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51a55182913d414b8cb7cf5eef9e1f3d" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 41515 } } peers { permanent_uuid: "91ace7d8479a4eb8abca1d23fa091e5f" member_type: VOTER last_known_addr { host: "127.20.113.61" port: 44499 } } peers { permanent_uuid: "a526ae086af444469aa58a0cb06523be" member_type: VOTER last_known_addr { host: "127.20.113.60" port: 34865 } } }
I20260806 09:31:52.801522 8376 sys_catalog.cc:458] T 00000000000000000000000000000000 P a526ae086af444469aa58a0cb06523be [sys.catalog]: This master's current role is: FOLLOWER
I20260806 09:31:52.838575 8426 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 09:31:52.843667 8426 catalog_manager.cc:1504] Loading token signing keys...
I20260806 09:31:52.878364 8426 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 51a55182913d414b8cb7cf5eef9e1f3d: Generated new TSK 0
I20260806 09:31:52.879575 8426 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 09:31:53.531844 8396 catalog_manager.cc:1259] Loaded cluster ID: f29cfc3e32b74cd084b529405e5ae619
I20260806 09:31:53.531956 8396 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a526ae086af444469aa58a0cb06523be: loading cluster ID for follower catalog manager: success
I20260806 09:31:53.534469 8396 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a526ae086af444469aa58a0cb06523be: acquiring CA information for follower catalog manager: success
I20260806 09:31:53.536311 8396 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a526ae086af444469aa58a0cb06523be: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 09:31:53.621546 8422 catalog_manager.cc:1259] Loaded cluster ID: f29cfc3e32b74cd084b529405e5ae619
I20260806 09:31:53.621655 8422 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 91ace7d8479a4eb8abca1d23fa091e5f: loading cluster ID for follower catalog manager: success
I20260806 09:31:53.624114 8422 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 91ace7d8479a4eb8abca1d23fa091e5f: acquiring CA information for follower catalog manager: success
I20260806 09:31:53.625929 8422 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 91ace7d8479a4eb8abca1d23fa091e5f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 09:31:54.028406 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 8039
I20260806 09:31:54.029773 8131 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:31:54.142974 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 8039
I20260806 09:31:54.174654 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 8146
I20260806 09:31:54.175994 8229 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:31:54.286145 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 8146
I20260806 09:31:54.313764 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 8246
I20260806 09:31:54.315011 8343 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:31:54.424437 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 8246
2026-08-06T09:31:54Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (5198 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786008495763054-20932-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 09:31:54 dist-test-slave-45dx krb5kdc[8451](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 09:31:54 dist-test-slave-45dx krb5kdc[8451](info): set up 2 sockets
Aug 06 09:31:54 dist-test-slave-45dx krb5kdc[8451](info): commencing operation
krb5kdc: starting...
W20260806 09:31:55.739086 20932 mini_kdc.cc:121] Time spent starting KDC: real 1.265s user 0.004s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786008495763054-20932-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786008495763054-20932-0/krb5kdc/test-user.keytab.
Aug 06 09:31:55 dist-test-slave-45dx krb5kdc[8451](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008715, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T09:31:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T09:31:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.113.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.113.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.113.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab.
Entry for principal kudu/127.20.113.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab.
I20260806 09:31:56.043983 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786008495763054-20932-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786008495763054-20932-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786008495763054-20932-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.113.62:42975
--webserver_interface=127.20.113.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab
--principal=kudu/127.20.113.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:33237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.113.62:42975
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:31:56.392655 8471 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:31:56.392935 8471 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:31:56.393024 8471 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:31:56.400033 8471 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 09:31:56.400131 8471 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:31:56.400177 8471 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:31:56.400233 8471 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 09:31:56.400283 8471 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 09:31:56.410409 8471 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:33237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786008495763054-20932-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786008495763054-20932-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.113.62:42975
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab
--principal=kudu/127.20.113.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.113.62:42975
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.113.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786008495763054-20932-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:31:56.411582 8471 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:31:56.412827 8471 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:31:56.424973 8477 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:31:56.424952 8483 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:31:56.424949 8481 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:31:56.425872 8471 server_base.cc:1034] running on GCE node
I20260806 09:31:56.426606 8471 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:31:56.427963 8471 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:31:56.429184 8471 hybrid_clock.cc:648] HybridClock initialized: now 1786008716429140 us; error 32 us; skew 500 ppm
I20260806 09:31:56.429556 8471 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:31:56 dist-test-slave-45dx krb5kdc[8451](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008716, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:31:56.439254 8471 init.cc:377] Logged in from keytab as kudu/127.20.113.62@KRBTEST.COM (short username kudu)
I20260806 09:31:56.440845 8471 webserver.cc:460] Webserver started at http://127.20.113.62:40327/ using document root <none> and password file <none>
I20260806 09:31:56.441524 8471 fs_manager.cc:359] Metadata directory not provided
I20260806 09:31:56.441673 8471 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:31:56.442034 8471 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:31:56.445055 8471 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786008495763054-20932-0/minicluster-data/master-0/data/instance:
uuid: "723fffad81194ca8a9f71f102afe3823"
format_stamp: "Formatted at 2026-08-06 09:31:56 on dist-test-slave-45dx"
server_key: "793e3b1904e318bbcf6e6958b2815143"
server_key_iv: "a484b7146fe500a1817632ca3a9ca4d5"
server_key_version: "encryptionkey@0"
I20260806 09:31:56.445801 8471 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786008495763054-20932-0/minicluster-data/master-0/wal/instance:
uuid: "723fffad81194ca8a9f71f102afe3823"
format_stamp: "Formatted at 2026-08-06 09:31:56 on dist-test-slave-45dx"
server_key: "793e3b1904e318bbcf6e6958b2815143"
server_key_iv: "a484b7146fe500a1817632ca3a9ca4d5"
server_key_version: "encryptionkey@0"
I20260806 09:31:56.450870 8471 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 09:31:56.454244 8498 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:31:56.455493 8471 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 09:31:56.455675 8471 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786008495763054-20932-0/minicluster-data/master-0/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786008495763054-20932-0/minicluster-data/master-0/wal
uuid: "723fffad81194ca8a9f71f102afe3823"
format_stamp: "Formatted at 2026-08-06 09:31:56 on dist-test-slave-45dx"
server_key: "793e3b1904e318bbcf6e6958b2815143"
server_key_iv: "a484b7146fe500a1817632ca3a9ca4d5"
server_key_version: "encryptionkey@0"
I20260806 09:31:56.455876 8471 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786008495763054-20932-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786008495763054-20932-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786008495763054-20932-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:31:56.473007 8471 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:31:56.477037 8471 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:31:56.477324 8471 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:31:56.496454 8471 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.62:42975
I20260806 09:31:56.496527 8573 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.62:42975 every 8 connection(s)
I20260806 09:31:56.497884 8471 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
I20260806 09:31:56.501590 8574 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:31:56.506492 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 8471
I20260806 09:31:56.506731 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786008495763054-20932-0/minicluster-data/master-0/wal/instance
I20260806 09:31:56.507570 20932 external_mini_cluster.cc:1428] Setting key 531411332ec93291e544437298ab7b69
I20260806 09:31:56.513149 8574 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 723fffad81194ca8a9f71f102afe3823: Bootstrap starting.
I20260806 09:31:56.516048 8574 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 723fffad81194ca8a9f71f102afe3823: Neither blocks nor log segments found. Creating new log.
I20260806 09:31:56.517076 8574 log.cc:824] T 00000000000000000000000000000000 P 723fffad81194ca8a9f71f102afe3823: Log is configured to *not* fsync() on all Append() calls
I20260806 09:31:56.520488 8574 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 723fffad81194ca8a9f71f102afe3823: No bootstrap required, opened a new log
Aug 06 09:31:56 dist-test-slave-45dx krb5kdc[8451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008715, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.113.62@KRBTEST.COM
I20260806 09:31:56.531021 8574 raft_consensus.cc:348] T 00000000000000000000000000000000 P 723fffad81194ca8a9f71f102afe3823 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "723fffad81194ca8a9f71f102afe3823" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 42975 } }
I20260806 09:31:56.531468 8574 raft_consensus.cc:374] T 00000000000000000000000000000000 P 723fffad81194ca8a9f71f102afe3823 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:31:56.531589 8574 raft_consensus.cc:729] T 00000000000000000000000000000000 P 723fffad81194ca8a9f71f102afe3823 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 723fffad81194ca8a9f71f102afe3823, State: Initialized, Role: FOLLOWER
I20260806 09:31:56.532112 8583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:31:56.509572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33172 (local address 127.20.113.62:42975)
0806 09:31:56.510370 (+ 798us) server_negotiation.cc:206] Beginning negotiation
0806 09:31:56.510380 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 09:31:56.510421 (+ 41us) server_negotiation.cc:407] Connection header received
0806 09:31:56.512119 (+ 1698us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:31:56.512141 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:31:56.512576 (+ 435us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:31:56.512938 (+ 362us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:31:56.514424 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:31:56.515655 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:31:56.517096 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:31:56.518161 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:31:56.518370 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:31:56.523368 (+ 4998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:31:56.523377 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:31:56.523395 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:31:56.523451 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:31:56.527541 (+ 4090us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:31:56.528286 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:31:56.528299 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:31:56.528328 (+ 29us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:31:56.528552 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:31:56.528946 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:31:56.528951 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:31:56.528955 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:31:56.529686 (+ 731us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:31:56.529994 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0806 09:31:56.530305 (+ 311us) server_negotiation.cc:299] Negotiation successful
0806 09:31:56.530730 (+ 425us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":577,"thread_start_us":350,"threads_started":1}
I20260806 09:31:56.532341 8574 consensus_queue.cc:260] T 00000000000000000000000000000000 P 723fffad81194ca8a9f71f102afe3823 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "723fffad81194ca8a9f71f102afe3823" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 42975 } }
I20260806 09:31:56.532652 8574 raft_consensus.cc:388] T 00000000000000000000000000000000 P 723fffad81194ca8a9f71f102afe3823 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 09:31:56.532810 8574 raft_consensus.cc:482] T 00000000000000000000000000000000 P 723fffad81194ca8a9f71f102afe3823 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 09:31:56.532996 8574 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 723fffad81194ca8a9f71f102afe3823 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:31:56.535684 8574 raft_consensus.cc:504] T 00000000000000000000000000000000 P 723fffad81194ca8a9f71f102afe3823 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "723fffad81194ca8a9f71f102afe3823" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 42975 } }
I20260806 09:31:56.536190 8574 leader_election.cc:302] T 00000000000000000000000000000000 P 723fffad81194ca8a9f71f102afe3823 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 723fffad81194ca8a9f71f102afe3823; no voters:
I20260806 09:31:56.536389 20932 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 09:31:56.536890 8574 leader_election.cc:288] T 00000000000000000000000000000000 P 723fffad81194ca8a9f71f102afe3823 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 09:31:56.537020 8585 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 723fffad81194ca8a9f71f102afe3823 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:31:56.538471 8585 raft_consensus.cc:686] T 00000000000000000000000000000000 P 723fffad81194ca8a9f71f102afe3823 [term 1 LEADER]: Becoming Leader. State: Replica: 723fffad81194ca8a9f71f102afe3823, State: Running, Role: LEADER
I20260806 09:31:56.539003 8574 sys_catalog.cc:564] T 00000000000000000000000000000000 P 723fffad81194ca8a9f71f102afe3823 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 09:31:56.539031 8585 consensus_queue.cc:237] T 00000000000000000000000000000000 P 723fffad81194ca8a9f71f102afe3823 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "723fffad81194ca8a9f71f102afe3823" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 42975 } }
I20260806 09:31:56.542294 8586 sys_catalog.cc:455] T 00000000000000000000000000000000 P 723fffad81194ca8a9f71f102afe3823 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "723fffad81194ca8a9f71f102afe3823" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "723fffad81194ca8a9f71f102afe3823" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 42975 } } }
I20260806 09:31:56.542778 8586 sys_catalog.cc:458] T 00000000000000000000000000000000 P 723fffad81194ca8a9f71f102afe3823 [sys.catalog]: This master's current role is: LEADER
I20260806 09:31:56.543150 8587 sys_catalog.cc:455] T 00000000000000000000000000000000 P 723fffad81194ca8a9f71f102afe3823 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 723fffad81194ca8a9f71f102afe3823. Latest consensus state: current_term: 1 leader_uuid: "723fffad81194ca8a9f71f102afe3823" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "723fffad81194ca8a9f71f102afe3823" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 42975 } } }
I20260806 09:31:56.543754 8587 sys_catalog.cc:458] T 00000000000000000000000000000000 P 723fffad81194ca8a9f71f102afe3823 [sys.catalog]: This master's current role is: LEADER
I20260806 09:31:56.549312 8594 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 09:31:56.553447 8594 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 09:31:56.561194 8594 catalog_manager.cc:1347] Generated new cluster ID: 9d6070c2c4cf43708f21b4b8af5d3819
I20260806 09:31:56.561303 8594 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 09:31:56.573383 8594 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 09:31:56.575253 8594 catalog_manager.cc:1504] Loading token signing keys...
I20260806 09:31:56.586218 8594 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 723fffad81194ca8a9f71f102afe3823: Generated new TSK 0
I20260806 09:31:56.587242 8594 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 09:31:56.598628 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 8471
I20260806 09:31:56.599833 8566 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:31:56.690104 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 8471
2026-08-06T09:31:56Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (2260 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 09:31:56 dist-test-slave-45dx krb5kdc[8615](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 09:31:56 dist-test-slave-45dx krb5kdc[8615](info): set up 2 sockets
Aug 06 09:31:56 dist-test-slave-45dx krb5kdc[8615](info): commencing operation
krb5kdc: starting...
W20260806 09:31:57.984019 20932 mini_kdc.cc:121] Time spent starting KDC: real 1.250s user 0.000s sys 0.047s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/krb5kdc/test-user.keytab.
Aug 06 09:31:58 dist-test-slave-45dx krb5kdc[8615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008718, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T09:31:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T09:31:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.113.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.113.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.113.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab.
Entry for principal kudu/127.20.113.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab.
I20260806 09:31:58.295954 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.113.62:44265
--webserver_interface=127.20.113.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab
--principal=kudu/127.20.113.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:46541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.113.62:44265
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:31:58.643016 8634 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:31:58.643349 8634 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:31:58.643507 8634 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:31:58.650733 8634 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 09:31:58.650875 8634 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:31:58.650977 8634 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:31:58.651078 8634 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 09:31:58.651152 8634 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 09:31:58.660997 8634 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:46541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.113.62:44265
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab
--principal=kudu/127.20.113.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.113.62:44265
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.113.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:31:58.662369 8634 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:31:58.663704 8634 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:31:58.676649 8641 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:31:58.676542 8642 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:31:58.676591 8645 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:31:58.677479 8634 server_base.cc:1034] running on GCE node
I20260806 09:31:58.678222 8634 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:31:58.679585 8634 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:31:58.680815 8634 hybrid_clock.cc:648] HybridClock initialized: now 1786008718680767 us; error 76 us; skew 500 ppm
I20260806 09:31:58.681152 8634 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:31:58 dist-test-slave-45dx krb5kdc[8615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008718, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:31:58.684195 8634 init.cc:377] Logged in from keytab as kudu/127.20.113.62@KRBTEST.COM (short username kudu)
I20260806 09:31:58.685642 8634 webserver.cc:460] Webserver started at http://127.20.113.62:34199/ using document root <none> and password file <none>
I20260806 09:31:58.686309 8634 fs_manager.cc:359] Metadata directory not provided
I20260806 09:31:58.686478 8634 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:31:58.686811 8634 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:31:58.689886 8634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/master-0/data/instance:
uuid: "2ebb3c6330eb44f9b537d7c3007f0271"
format_stamp: "Formatted at 2026-08-06 09:31:58 on dist-test-slave-45dx"
server_key: "05ba32b4bb156010031014cc0f150aea"
server_key_iv: "9dbb2824c88d4631079ed6f5d6310224"
server_key_version: "encryptionkey@0"
I20260806 09:31:58.690666 8634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/master-0/wal/instance:
uuid: "2ebb3c6330eb44f9b537d7c3007f0271"
format_stamp: "Formatted at 2026-08-06 09:31:58 on dist-test-slave-45dx"
server_key: "05ba32b4bb156010031014cc0f150aea"
server_key_iv: "9dbb2824c88d4631079ed6f5d6310224"
server_key_version: "encryptionkey@0"
I20260806 09:31:58.695652 8634 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.006s
I20260806 09:31:58.698997 8660 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:31:58.700485 8634 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 09:31:58.700654 8634 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/master-0/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/master-0/wal
uuid: "2ebb3c6330eb44f9b537d7c3007f0271"
format_stamp: "Formatted at 2026-08-06 09:31:58 on dist-test-slave-45dx"
server_key: "05ba32b4bb156010031014cc0f150aea"
server_key_iv: "9dbb2824c88d4631079ed6f5d6310224"
server_key_version: "encryptionkey@0"
I20260806 09:31:58.700892 8634 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:31:58.712940 8634 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:31:58.716953 8634 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:31:58.717245 8634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:31:58.735870 8634 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.62:44265
I20260806 09:31:58.735924 8731 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.62:44265 every 8 connection(s)
I20260806 09:31:58.737280 8634 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
I20260806 09:31:58.737728 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 8634
I20260806 09:31:58.737979 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/master-0/wal/instance
I20260806 09:31:58.738910 20932 external_mini_cluster.cc:1428] Setting key 2f90189e913f4a3a293a3ee6253f20c0
I20260806 09:31:58.741928 8732 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 06 09:31:58 dist-test-slave-45dx krb5kdc[8615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008718, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.113.62@KRBTEST.COM
I20260806 09:31:58.754992 8732 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ebb3c6330eb44f9b537d7c3007f0271: Bootstrap starting.
I20260806 09:31:58.758345 8732 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2ebb3c6330eb44f9b537d7c3007f0271: Neither blocks nor log segments found. Creating new log.
I20260806 09:31:58.759425 8732 log.cc:824] T 00000000000000000000000000000000 P 2ebb3c6330eb44f9b537d7c3007f0271: Log is configured to *not* fsync() on all Append() calls
I20260806 09:31:58.762219 8736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:31:58.741051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53162 (local address 127.20.113.62:44265)
0806 09:31:58.741857 (+ 806us) server_negotiation.cc:206] Beginning negotiation
0806 09:31:58.741868 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 09:31:58.741909 (+ 41us) server_negotiation.cc:407] Connection header received
0806 09:31:58.743504 (+ 1595us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:31:58.743524 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:31:58.743865 (+ 341us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:31:58.744174 (+ 309us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:31:58.745987 (+ 1813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:31:58.747302 (+ 1315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:31:58.749139 (+ 1837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:31:58.749757 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:31:58.749885 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:31:58.754758 (+ 4873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:31:58.754766 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:31:58.754779 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:31:58.754817 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:31:58.757841 (+ 3024us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:31:58.758538 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:31:58.758545 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:31:58.758551 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:31:58.758725 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:31:58.759135 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:31:58.759139 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:31:58.759141 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:31:58.759674 (+ 533us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:31:58.760012 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0806 09:31:58.760495 (+ 483us) server_negotiation.cc:299] Negotiation successful
0806 09:31:58.760924 (+ 429us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":601,"thread_start_us":385,"threads_started":1}
I20260806 09:31:58.762822 8732 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ebb3c6330eb44f9b537d7c3007f0271: No bootstrap required, opened a new log
I20260806 09:31:58.770635 8732 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2ebb3c6330eb44f9b537d7c3007f0271 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ebb3c6330eb44f9b537d7c3007f0271" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 44265 } }
I20260806 09:31:58.770972 8732 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2ebb3c6330eb44f9b537d7c3007f0271 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:31:58.771046 8732 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2ebb3c6330eb44f9b537d7c3007f0271 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ebb3c6330eb44f9b537d7c3007f0271, State: Initialized, Role: FOLLOWER
I20260806 09:31:58.771587 8732 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2ebb3c6330eb44f9b537d7c3007f0271 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ebb3c6330eb44f9b537d7c3007f0271" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 44265 } }
I20260806 09:31:58.771800 8732 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2ebb3c6330eb44f9b537d7c3007f0271 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 09:31:58.771895 8732 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2ebb3c6330eb44f9b537d7c3007f0271 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 09:31:58.772027 8732 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2ebb3c6330eb44f9b537d7c3007f0271 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:31:58.774300 8732 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2ebb3c6330eb44f9b537d7c3007f0271 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ebb3c6330eb44f9b537d7c3007f0271" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 44265 } }
I20260806 09:31:58.774818 8732 leader_election.cc:302] T 00000000000000000000000000000000 P 2ebb3c6330eb44f9b537d7c3007f0271 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2ebb3c6330eb44f9b537d7c3007f0271; no voters:
I20260806 09:31:58.775417 8732 leader_election.cc:288] T 00000000000000000000000000000000 P 2ebb3c6330eb44f9b537d7c3007f0271 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 09:31:58.775573 8738 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2ebb3c6330eb44f9b537d7c3007f0271 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:31:58.776291 8738 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2ebb3c6330eb44f9b537d7c3007f0271 [term 1 LEADER]: Becoming Leader. State: Replica: 2ebb3c6330eb44f9b537d7c3007f0271, State: Running, Role: LEADER
I20260806 09:31:58.776868 8738 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2ebb3c6330eb44f9b537d7c3007f0271 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ebb3c6330eb44f9b537d7c3007f0271" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 44265 } }
I20260806 09:31:58.777422 8732 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2ebb3c6330eb44f9b537d7c3007f0271 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 09:31:58.779897 8740 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ebb3c6330eb44f9b537d7c3007f0271 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2ebb3c6330eb44f9b537d7c3007f0271" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ebb3c6330eb44f9b537d7c3007f0271" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 44265 } } }
I20260806 09:31:58.780395 8740 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ebb3c6330eb44f9b537d7c3007f0271 [sys.catalog]: This master's current role is: LEADER
I20260806 09:31:58.779932 8741 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ebb3c6330eb44f9b537d7c3007f0271 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2ebb3c6330eb44f9b537d7c3007f0271. Latest consensus state: current_term: 1 leader_uuid: "2ebb3c6330eb44f9b537d7c3007f0271" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ebb3c6330eb44f9b537d7c3007f0271" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 44265 } } }
I20260806 09:31:58.781057 8741 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ebb3c6330eb44f9b537d7c3007f0271 [sys.catalog]: This master's current role is: LEADER
I20260806 09:31:58.785017 8746 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 09:31:58.789582 8746 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 09:31:58.798043 8746 catalog_manager.cc:1347] Generated new cluster ID: 77281e58ac894f2c99060d83083d2db4
I20260806 09:31:58.798149 8746 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 09:31:58.824141 8746 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 09:31:58.826100 8746 catalog_manager.cc:1504] Loading token signing keys...
I20260806 09:31:58.837316 8746 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2ebb3c6330eb44f9b537d7c3007f0271: Generated new TSK 0
I20260806 09:31:58.838294 8746 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.113.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.113.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.113.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.1.keytab.
Entry for principal kudu/127.20.113.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.1.keytab.
I20260806 09:31:58.920526 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.113.1:0
--local_ip_for_outbound_sockets=127.20.113.1
--webserver_interface=127.20.113.1
--webserver_port=0
--tserver_master_addrs=127.20.113.62:44265
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.1.keytab
--principal=kudu/127.20.113.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:46541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:31:59.271772 8770 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:31:59.272128 8770 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:31:59.272292 8770 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:31:59.279794 8770 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:31:59.279901 8770 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:31:59.280016 8770 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.113.1
I20260806 09:31:59.290181 8770 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:46541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.1.keytab
--principal=kudu/127.20.113.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.113.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.113.1
--webserver_port=0
--tserver_master_addrs=127.20.113.62:44265
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.113.1
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:31:59.291466 8770 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:31:59.292888 8770 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:31:59.306367 8782 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:31:59.306416 8777 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:31:59.306371 8778 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:31:59.307590 8770 server_base.cc:1034] running on GCE node
I20260806 09:31:59.308193 8770 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:31:59.309166 8770 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:31:59.310405 8770 hybrid_clock.cc:648] HybridClock initialized: now 1786008719310355 us; error 43 us; skew 500 ppm
I20260806 09:31:59.310732 8770 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:31:59 dist-test-slave-45dx krb5kdc[8615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008719, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:31:59.314040 8770 init.cc:377] Logged in from keytab as kudu/127.20.113.1@KRBTEST.COM (short username kudu)
I20260806 09:31:59.315445 8770 webserver.cc:460] Webserver started at http://127.20.113.1:45327/ using document root <none> and password file <none>
I20260806 09:31:59.316138 8770 fs_manager.cc:359] Metadata directory not provided
I20260806 09:31:59.316286 8770 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:31:59.316648 8770 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:31:59.319675 8770 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-0/data/instance:
uuid: "5425fb70daaf4f5e8bf84f5e5143e89f"
format_stamp: "Formatted at 2026-08-06 09:31:59 on dist-test-slave-45dx"
server_key: "29d545bab1cc17611d12fb3650b096a5"
server_key_iv: "96d0310e72ed5b248b2fbe011b5f80c5"
server_key_version: "encryptionkey@0"
I20260806 09:31:59.320547 8770 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-0/wal/instance:
uuid: "5425fb70daaf4f5e8bf84f5e5143e89f"
format_stamp: "Formatted at 2026-08-06 09:31:59 on dist-test-slave-45dx"
server_key: "29d545bab1cc17611d12fb3650b096a5"
server_key_iv: "96d0310e72ed5b248b2fbe011b5f80c5"
server_key_version: "encryptionkey@0"
I20260806 09:31:59.325644 8770 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260806 09:31:59.329144 8790 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:31:59.330432 8770 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 09:31:59.330611 8770 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-0/wal
uuid: "5425fb70daaf4f5e8bf84f5e5143e89f"
format_stamp: "Formatted at 2026-08-06 09:31:59 on dist-test-slave-45dx"
server_key: "29d545bab1cc17611d12fb3650b096a5"
server_key_iv: "96d0310e72ed5b248b2fbe011b5f80c5"
server_key_version: "encryptionkey@0"
I20260806 09:31:59.330826 8770 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:31:59.347874 8770 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:31:59.351795 8770 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:31:59.352108 8770 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:31:59.353132 8770 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:31:59.354460 8770 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:31:59.354563 8770 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:31:59.354720 8770 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:31:59.354809 8770 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:31:59.390237 8770 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.1:46039
I20260806 09:31:59.390285 8946 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.1:46039 every 8 connection(s)
I20260806 09:31:59.391511 8770 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-0/data/info.pb
I20260806 09:31:59.393324 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 8770
I20260806 09:31:59.393564 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-0/wal/instance
I20260806 09:31:59.394328 20932 external_mini_cluster.cc:1428] Setting key 03ff6f909be63d4b3738d11c7a9abc8f
Aug 06 09:31:59 dist-test-slave-45dx krb5kdc[8615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008719, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.1@KRBTEST.COM for kudu/127.20.113.62@KRBTEST.COM
I20260806 09:31:59.413697 8951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:31:59.394384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.1:34687 (local address 127.20.113.62:44265)
0806 09:31:59.394933 (+ 549us) server_negotiation.cc:206] Beginning negotiation
0806 09:31:59.394939 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:31:59.396101 (+ 1162us) server_negotiation.cc:407] Connection header received
0806 09:31:59.398086 (+ 1985us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:31:59.398094 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:31:59.398182 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:31:59.398301 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:31:59.399746 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:31:59.400859 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:31:59.402423 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:31:59.403126 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:31:59.403273 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:31:59.406751 (+ 3478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:31:59.406760 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:31:59.406768 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:31:59.406811 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:31:59.410470 (+ 3659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:31:59.411315 (+ 845us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:31:59.411323 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:31:59.411328 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:31:59.411480 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:31:59.411935 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:31:59.411940 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:31:59.411944 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:31:59.412235 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:31:59.412418 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0806 09:31:59.413099 (+ 681us) server_negotiation.cc:299] Negotiation successful
0806 09:31:59.413332 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":257,"threads_started":1}
I20260806 09:31:59.414587 8952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:31:59.395036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44265 (local address 127.20.113.1:34687)
0806 09:31:59.395928 (+ 892us) negotiation.cc:107] Waiting for socket to connect
0806 09:31:59.395979 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 09:31:59.397826 (+ 1847us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:31:59.398511 (+ 685us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:31:59.398518 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:31:59.399013 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:31:59.399576 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:31:59.399593 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:31:59.401020 (+ 1427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:31:59.401024 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:31:59.402249 (+ 1225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:31:59.402262 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:31:59.403344 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:31:59.403349 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:31:59.403537 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:31:59.404242 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:31:59.404271 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:31:59.406550 (+ 2279us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:31:59.410611 (+ 4061us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:31:59.410621 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:31:59.410637 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:31:59.411198 (+ 561us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:31:59.411607 (+ 409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:31:59.411612 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:31:59.411616 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:31:59.411793 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:31:59.412392 (+ 599us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:31:59.412400 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:31:59.412724 (+ 324us) client_negotiation.cc:769] Sending connection context
0806 09:31:59.412990 (+ 266us) client_negotiation.cc:240] Negotiation successful
0806 09:31:59.413499 (+ 509us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":609,"thread_start_us":360,"threads_started":1}
I20260806 09:31:59.416137 8947 heartbeater.cc:347] Connected to a master server at 127.20.113.62:44265
I20260806 09:31:59.416498 8947 heartbeater.cc:464] Registering TS with master...
I20260806 09:31:59.417443 8947 heartbeater.cc:511] Master 127.20.113.62:44265 requested a full tablet report, sending...
I20260806 09:31:59.419979 8685 ts_manager.cc:194] Registered new tserver with Master: 5425fb70daaf4f5e8bf84f5e5143e89f (127.20.113.1:46039)
I20260806 09:31:59.422155 8685 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.113.1@KRBTEST.COM'} at 127.20.113.1:34687
WARNING: no policy specified for kudu/127.20.113.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.113.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.113.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.2.keytab.
Entry for principal kudu/127.20.113.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.2.keytab.
I20260806 09:31:59.468617 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.113.2:0
--local_ip_for_outbound_sockets=127.20.113.2
--webserver_interface=127.20.113.2
--webserver_port=0
--tserver_master_addrs=127.20.113.62:44265
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.2.keytab
--principal=kudu/127.20.113.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:46541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:31:59.818011 8956 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:31:59.818285 8956 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:31:59.818507 8956 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:31:59.825601 8956 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:31:59.825733 8956 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:31:59.825941 8956 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.113.2
I20260806 09:31:59.835692 8956 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:46541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.2.keytab
--principal=kudu/127.20.113.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.113.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.113.2
--webserver_port=0
--tserver_master_addrs=127.20.113.62:44265
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.113.2
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:31:59.836982 8956 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:31:59.838305 8956 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:31:59.851267 8965 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:31:59.851200 8966 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:31:59.851243 8968 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:31:59.852264 8956 server_base.cc:1034] running on GCE node
I20260806 09:31:59.852897 8956 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:31:59.853768 8956 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:31:59.854972 8956 hybrid_clock.cc:648] HybridClock initialized: now 1786008719854930 us; error 40 us; skew 500 ppm
I20260806 09:31:59.855248 8956 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:31:59 dist-test-slave-45dx krb5kdc[8615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008719, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:31:59.858486 8956 init.cc:377] Logged in from keytab as kudu/127.20.113.2@KRBTEST.COM (short username kudu)
I20260806 09:31:59.860018 8956 webserver.cc:460] Webserver started at http://127.20.113.2:45355/ using document root <none> and password file <none>
I20260806 09:31:59.860788 8956 fs_manager.cc:359] Metadata directory not provided
I20260806 09:31:59.860941 8956 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:31:59.861274 8956 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:31:59.864274 8956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-1/data/instance:
uuid: "7702d30841eb48b79a91eaaeaf45e426"
format_stamp: "Formatted at 2026-08-06 09:31:59 on dist-test-slave-45dx"
server_key: "b4b9ffe9ba6a109c233ab2bf8d6dc65e"
server_key_iv: "31395436d915c43eeceff885ef75b388"
server_key_version: "encryptionkey@0"
I20260806 09:31:59.865161 8956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-1/wal/instance:
uuid: "7702d30841eb48b79a91eaaeaf45e426"
format_stamp: "Formatted at 2026-08-06 09:31:59 on dist-test-slave-45dx"
server_key: "b4b9ffe9ba6a109c233ab2bf8d6dc65e"
server_key_iv: "31395436d915c43eeceff885ef75b388"
server_key_version: "encryptionkey@0"
I20260806 09:31:59.870638 8956 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 09:31:59.874109 8980 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:31:59.875389 8956 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 09:31:59.875571 8956 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-1/wal
uuid: "7702d30841eb48b79a91eaaeaf45e426"
format_stamp: "Formatted at 2026-08-06 09:31:59 on dist-test-slave-45dx"
server_key: "b4b9ffe9ba6a109c233ab2bf8d6dc65e"
server_key_iv: "31395436d915c43eeceff885ef75b388"
server_key_version: "encryptionkey@0"
I20260806 09:31:59.875761 8956 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:31:59.903972 8956 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:31:59.908731 8956 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:31:59.909034 8956 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:31:59.910048 8956 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:31:59.911434 8956 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:31:59.911510 8956 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:31:59.911660 8956 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:31:59.911744 8956 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:31:59.945855 8956 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.2:37455
I20260806 09:31:59.945914 9151 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.2:37455 every 8 connection(s)
I20260806 09:31:59.947191 8956 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-1/data/info.pb
I20260806 09:31:59.948719 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 8956
I20260806 09:31:59.948939 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-1/wal/instance
I20260806 09:31:59.949584 20932 external_mini_cluster.cc:1428] Setting key 9e93d5c390403ab609109895a747ec74
Aug 06 09:31:59 dist-test-slave-45dx krb5kdc[8615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008719, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.2@KRBTEST.COM for kudu/127.20.113.62@KRBTEST.COM
I20260806 09:31:59.967392 9154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:31:59.949978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.2:47925 (local address 127.20.113.62:44265)
0806 09:31:59.950471 (+ 493us) server_negotiation.cc:206] Beginning negotiation
0806 09:31:59.950477 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:31:59.951605 (+ 1128us) server_negotiation.cc:407] Connection header received
0806 09:31:59.953550 (+ 1945us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:31:59.953555 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:31:59.953647 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:31:59.953763 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:31:59.955236 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:31:59.955982 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:31:59.957583 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:31:59.958005 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:31:59.958157 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:31:59.961669 (+ 3512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:31:59.961674 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:31:59.961677 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:31:59.961707 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:31:59.964284 (+ 2577us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:31:59.965090 (+ 806us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:31:59.965095 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:31:59.965098 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:31:59.965231 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:31:59.965681 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:31:59.965685 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:31:59.965687 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:31:59.965937 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:31:59.966095 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0806 09:31:59.966779 (+ 684us) server_negotiation.cc:299] Negotiation successful
0806 09:31:59.967018 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":248,"threads_started":1}
I20260806 09:31:59.968398 9155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:31:59.950655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44265 (local address 127.20.113.2:47925)
0806 09:31:59.951442 (+ 787us) negotiation.cc:107] Waiting for socket to connect
0806 09:31:59.951510 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0806 09:31:59.953292 (+ 1782us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:31:59.954009 (+ 717us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:31:59.954018 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:31:59.954500 (+ 482us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:31:59.955067 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:31:59.955084 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:31:59.956142 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:31:59.956147 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:31:59.957438 (+ 1291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:31:59.957450 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:31:59.958265 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:31:59.958273 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:31:59.958499 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:31:59.959273 (+ 774us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:31:59.959295 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:31:59.961457 (+ 2162us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:31:59.964489 (+ 3032us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:31:59.964497 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:31:59.964512 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:31:59.964941 (+ 429us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:31:59.965347 (+ 406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:31:59.965352 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:31:59.965356 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:31:59.965552 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:31:59.966070 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:31:59.966078 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:31:59.966410 (+ 332us) client_negotiation.cc:769] Sending connection context
0806 09:31:59.966656 (+ 246us) client_negotiation.cc:240] Negotiation successful
0806 09:31:59.967197 (+ 541us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":531,"thread_start_us":330,"threads_started":1}
I20260806 09:31:59.969823 9152 heartbeater.cc:347] Connected to a master server at 127.20.113.62:44265
I20260806 09:31:59.970129 9152 heartbeater.cc:464] Registering TS with master...
I20260806 09:31:59.970831 9152 heartbeater.cc:511] Master 127.20.113.62:44265 requested a full tablet report, sending...
I20260806 09:31:59.972461 8685 ts_manager.cc:194] Registered new tserver with Master: 7702d30841eb48b79a91eaaeaf45e426 (127.20.113.2:37455)
I20260806 09:31:59.973277 8685 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.113.2@KRBTEST.COM'} at 127.20.113.2:47925
WARNING: no policy specified for kudu/127.20.113.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.113.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.113.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.3.keytab.
Entry for principal kudu/127.20.113.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.3.keytab.
I20260806 09:32:00.025408 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.113.3:0
--local_ip_for_outbound_sockets=127.20.113.3
--webserver_interface=127.20.113.3
--webserver_port=0
--tserver_master_addrs=127.20.113.62:44265
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.3.keytab
--principal=kudu/127.20.113.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:46541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:32:00.379707 9158 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:32:00.380061 9158 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:32:00.380240 9158 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:32:00.387722 9158 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:32:00.387881 9158 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:32:00.388079 9158 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.113.3
I20260806 09:32:00.397712 9158 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:46541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.3.keytab
--principal=kudu/127.20.113.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.113.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.113.3
--webserver_port=0
--tserver_master_addrs=127.20.113.62:44265
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.113.3
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:32:00.399004 9158 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:32:00.400480 9158 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:32:00.413831 9165 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:32:00.414042 9164 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:32:00.414072 9167 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:32:00.414942 9158 server_base.cc:1034] running on GCE node
I20260806 09:32:00.415658 9158 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:32:00.416687 9158 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:32:00.417944 9158 hybrid_clock.cc:648] HybridClock initialized: now 1786008720417904 us; error 49 us; skew 500 ppm
I20260806 09:32:00.418303 9158 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:32:00 dist-test-slave-45dx krb5kdc[8615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008720, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:32:00.421833 9158 init.cc:377] Logged in from keytab as kudu/127.20.113.3@KRBTEST.COM (short username kudu)
I20260806 09:32:00.423465 9158 webserver.cc:460] Webserver started at http://127.20.113.3:34351/ using document root <none> and password file <none>
I20260806 09:32:00.424194 9158 fs_manager.cc:359] Metadata directory not provided
I20260806 09:32:00.424409 9158 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:32:00.424783 9158 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:32:00.425885 8947 heartbeater.cc:503] Master 127.20.113.62:44265 was elected leader, sending a full tablet report...
I20260806 09:32:00.428220 9158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-2/data/instance:
uuid: "339f163711b143c6a6a210b47915fb59"
format_stamp: "Formatted at 2026-08-06 09:32:00 on dist-test-slave-45dx"
server_key: "e8f9c570bff6a1289f44f97f65712e3a"
server_key_iv: "706b241d6e9475c962eac2b2696c4650"
server_key_version: "encryptionkey@0"
I20260806 09:32:00.429126 9158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-2/wal/instance:
uuid: "339f163711b143c6a6a210b47915fb59"
format_stamp: "Formatted at 2026-08-06 09:32:00 on dist-test-slave-45dx"
server_key: "e8f9c570bff6a1289f44f97f65712e3a"
server_key_iv: "706b241d6e9475c962eac2b2696c4650"
server_key_version: "encryptionkey@0"
I20260806 09:32:00.434684 9158 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 09:32:00.438217 9175 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:32:00.439530 9158 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260806 09:32:00.439718 9158 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-2/wal
uuid: "339f163711b143c6a6a210b47915fb59"
format_stamp: "Formatted at 2026-08-06 09:32:00 on dist-test-slave-45dx"
server_key: "e8f9c570bff6a1289f44f97f65712e3a"
server_key_iv: "706b241d6e9475c962eac2b2696c4650"
server_key_version: "encryptionkey@0"
I20260806 09:32:00.439929 9158 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:32:00.457647 9158 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:32:00.462301 9158 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:32:00.462626 9158 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:32:00.463660 9158 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:32:00.465107 9158 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:32:00.465214 9158 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:32:00.465371 9158 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:32:00.465447 9158 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:32:00.500284 9158 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.3:39849
I20260806 09:32:00.500379 9349 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.3:39849 every 8 connection(s)
I20260806 09:32:00.501648 9158 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-2/data/info.pb
I20260806 09:32:00.509968 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 9158
I20260806 09:32:00.510159 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-2/wal/instance
I20260806 09:32:00.510679 20932 external_mini_cluster.cc:1428] Setting key c2d3ef5a95dc8b02b56ed3554f5b0410
Aug 06 09:32:00 dist-test-slave-45dx krb5kdc[8615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008720, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.3@KRBTEST.COM for kudu/127.20.113.62@KRBTEST.COM
I20260806 09:32:00.523154 9353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:00.504258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.3:38225 (local address 127.20.113.62:44265)
0806 09:32:00.504827 (+ 569us) server_negotiation.cc:206] Beginning negotiation
0806 09:32:00.504836 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 09:32:00.505765 (+ 929us) server_negotiation.cc:407] Connection header received
0806 09:32:00.507633 (+ 1868us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:32:00.507639 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:32:00.507732 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:32:00.507871 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:32:00.509482 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:00.510446 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:00.512549 (+ 2103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:00.513054 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:00.513198 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:32:00.517046 (+ 3848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:32:00.517051 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:32:00.517055 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:32:00.517091 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:32:00.519720 (+ 2629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:32:00.520605 (+ 885us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:32:00.520612 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:32:00.520615 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:32:00.520760 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:32:00.521244 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:32:00.521250 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:32:00.521253 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:32:00.521552 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:32:00.521724 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0806 09:32:00.522500 (+ 776us) server_negotiation.cc:299] Negotiation successful
0806 09:32:00.522740 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":408,"thread_start_us":264,"threads_started":1}
I20260806 09:32:00.524191 9354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:00.504861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44265 (local address 127.20.113.3:38225)
0806 09:32:00.505587 (+ 726us) negotiation.cc:107] Waiting for socket to connect
0806 09:32:00.505626 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 09:32:00.507339 (+ 1713us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:32:00.508141 (+ 802us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:32:00.508151 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:32:00.508703 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:32:00.509300 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:32:00.509318 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:32:00.510616 (+ 1298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:32:00.510620 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:32:00.512397 (+ 1777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:32:00.512411 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:32:00.513303 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:32:00.513310 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:32:00.513554 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:32:00.514406 (+ 852us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:32:00.514435 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:32:00.516761 (+ 2326us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:32:00.519890 (+ 3129us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:32:00.519900 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:32:00.519921 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:32:00.520470 (+ 549us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:32:00.520896 (+ 426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:32:00.520901 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:32:00.520905 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:32:00.521116 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:32:00.521699 (+ 583us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:32:00.521707 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:32:00.522099 (+ 392us) client_negotiation.cc:769] Sending connection context
0806 09:32:00.522389 (+ 290us) client_negotiation.cc:240] Negotiation successful
0806 09:32:00.522981 (+ 592us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":496,"thread_start_us":293,"threads_started":1}
I20260806 09:32:00.525776 9350 heartbeater.cc:347] Connected to a master server at 127.20.113.62:44265
I20260806 09:32:00.526113 9350 heartbeater.cc:464] Registering TS with master...
I20260806 09:32:00.526979 9350 heartbeater.cc:511] Master 127.20.113.62:44265 requested a full tablet report, sending...
I20260806 09:32:00.528487 8683 ts_manager.cc:194] Registered new tserver with Master: 339f163711b143c6a6a210b47915fb59 (127.20.113.3:39849)
I20260806 09:32:00.529388 8683 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.113.3@KRBTEST.COM'} at 127.20.113.3:38225
I20260806 09:32:00.538779 20932 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 09:32:00.555665 9353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:00.543474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53168 (local address 127.20.113.62:44265)
0806 09:32:00.543628 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0806 09:32:00.543633 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:32:00.544201 (+ 568us) server_negotiation.cc:407] Connection header received
0806 09:32:00.544399 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:32:00.544403 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:32:00.544463 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:32:00.544574 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:32:00.545927 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:00.546646 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:00.547848 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:00.548245 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:00.548420 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:32:00.550803 (+ 2383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:32:00.550808 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:32:00.550811 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:32:00.550841 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:32:00.553444 (+ 2603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:32:00.554024 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:32:00.554027 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:32:00.554029 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:32:00.554145 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:32:00.554471 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:32:00.554474 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:32:00.554476 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:32:00.554775 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:32:00.554957 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0806 09:32:00.555263 (+ 306us) server_negotiation.cc:299] Negotiation successful
0806 09:32:00.555444 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 09:32:00.559625 8683 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53168:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 09:32:00.566571 8683 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 09:32:00.606127 9373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:00.580393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44444 (local address 127.20.113.1:46039)
0806 09:32:00.596948 (+ 16555us) server_negotiation.cc:206] Beginning negotiation
0806 09:32:00.596960 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 09:32:00.597700 (+ 740us) server_negotiation.cc:407] Connection header received
0806 09:32:00.597809 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:32:00.597817 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:32:00.598044 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:32:00.598262 (+ 218us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:32:00.599505 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:00.600635 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:00.603065 (+ 2430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:00.604375 (+ 1310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:00.604606 (+ 231us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:32:00.604851 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0806 09:32:00.605179 (+ 328us) server_negotiation.cc:299] Negotiation successful
0806 09:32:00.605753 (+ 574us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16341,"thread_start_us":16188,"threads_started":1}
I20260806 09:32:00.606678 9372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:00.579649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.1:46039 (local address 127.0.0.1:44444)
0806 09:32:00.597411 (+ 17762us) negotiation.cc:107] Waiting for socket to connect
0806 09:32:00.597433 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 09:32:00.597563 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:32:00.598563 (+ 1000us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:32:00.598567 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:32:00.598592 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:32:00.599321 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:32:00.599332 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:32:00.600770 (+ 1438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:32:00.600775 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:32:00.602902 (+ 2127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:32:00.602922 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:32:00.604699 (+ 1777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:32:00.604705 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:32:00.604916 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:32:00.604977 (+ 61us) client_negotiation.cc:769] Sending connection context
0806 09:32:00.605115 (+ 138us) client_negotiation.cc:240] Negotiation successful
0806 09:32:00.605351 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17629,"thread_start_us":17488,"threads_started":1}
I20260806 09:32:00.607338 9368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:00.579002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46132 (local address 127.20.113.3:39849)
0806 09:32:00.597854 (+ 18852us) server_negotiation.cc:206] Beginning negotiation
0806 09:32:00.597861 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:32:00.597889 (+ 28us) server_negotiation.cc:407] Connection header received
0806 09:32:00.597978 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:32:00.597984 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:32:00.598192 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:32:00.599392 (+ 1200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:32:00.599719 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:00.601312 (+ 1593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:00.603929 (+ 2617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:00.605396 (+ 1467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:00.605525 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:32:00.605761 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0806 09:32:00.606812 (+ 1051us) server_negotiation.cc:299] Negotiation successful
0806 09:32:00.606990 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18676,"thread_start_us":18520,"threads_started":1}
I20260806 09:32:00.607839 9370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:00.579085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.2:37455 (local address 127.0.0.1:53736)
0806 09:32:00.596515 (+ 17430us) negotiation.cc:107] Waiting for socket to connect
0806 09:32:00.596538 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 09:32:00.596690 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:32:00.598538 (+ 1848us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:32:00.598541 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:32:00.598566 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:32:00.599106 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:32:00.599117 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:32:00.601147 (+ 2030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:32:00.601152 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:32:00.603706 (+ 2554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:32:00.603727 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:32:00.605985 (+ 2258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:32:00.605991 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:32:00.606182 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:32:00.606446 (+ 264us) client_negotiation.cc:769] Sending connection context
0806 09:32:00.606561 (+ 115us) client_negotiation.cc:240] Negotiation successful
0806 09:32:00.606654 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17271,"mutex_wait_us":307,"spinlock_wait_cycles":2432,"thread_start_us":17070,"threads_started":1}
I20260806 09:32:00.608148 9371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:00.579656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53736 (local address 127.20.113.2:37455)
0806 09:32:00.595969 (+ 16313us) server_negotiation.cc:206] Beginning negotiation
0806 09:32:00.595983 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 09:32:00.596848 (+ 865us) server_negotiation.cc:407] Connection header received
0806 09:32:00.596994 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:32:00.597006 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:32:00.597957 (+ 951us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:32:00.598173 (+ 216us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:32:00.599276 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:00.600971 (+ 1695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:00.603883 (+ 2912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:00.605439 (+ 1556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:00.605589 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:32:00.605777 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0806 09:32:00.606943 (+ 1166us) server_negotiation.cc:299] Negotiation successful
0806 09:32:00.607146 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16073,"thread_start_us":15876,"threads_started":1}
I20260806 09:32:00.607048 9367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:00.578753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.3:39849 (local address 127.0.0.1:46132)
0806 09:32:00.593015 (+ 14262us) negotiation.cc:107] Waiting for socket to connect
0806 09:32:00.593084 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0806 09:32:00.593320 (+ 236us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:32:00.598449 (+ 5129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:32:00.598456 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:32:00.598485 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:32:00.599107 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:32:00.599127 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:32:00.601474 (+ 2347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:32:00.601480 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:32:00.603760 (+ 2280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:32:00.603772 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:32:00.606032 (+ 2260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:32:00.606040 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:32:00.606299 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:32:00.606331 (+ 32us) client_negotiation.cc:769] Sending connection context
0806 09:32:00.606424 (+ 93us) client_negotiation.cc:240] Negotiation successful
0806 09:32:00.606566 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13906,"mutex_wait_us":31,"thread_start_us":14765,"threads_started":1}
I20260806 09:32:00.611347 9249 tablet_service.cc:1467] Processing CreateTablet for tablet 25ec81213ad0498c9b48dd80220aeb46 (DEFAULT_TABLE table=test-table [id=0aa5c908cf74404386a300ea42a3898a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:32:00.611205 8855 tablet_service.cc:1467] Processing CreateTablet for tablet 25ec81213ad0498c9b48dd80220aeb46 (DEFAULT_TABLE table=test-table [id=0aa5c908cf74404386a300ea42a3898a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:32:00.611699 9050 tablet_service.cc:1467] Processing CreateTablet for tablet 25ec81213ad0498c9b48dd80220aeb46 (DEFAULT_TABLE table=test-table [id=0aa5c908cf74404386a300ea42a3898a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:32:00.612687 9249 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 25ec81213ad0498c9b48dd80220aeb46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:32:00.612756 8855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 25ec81213ad0498c9b48dd80220aeb46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:32:00.613027 9050 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 25ec81213ad0498c9b48dd80220aeb46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:32:00.625109 9374 tablet_bootstrap.cc:492] T 25ec81213ad0498c9b48dd80220aeb46 P 339f163711b143c6a6a210b47915fb59: Bootstrap starting.
I20260806 09:32:00.627542 9376 tablet_bootstrap.cc:492] T 25ec81213ad0498c9b48dd80220aeb46 P 7702d30841eb48b79a91eaaeaf45e426: Bootstrap starting.
I20260806 09:32:00.628346 9375 tablet_bootstrap.cc:492] T 25ec81213ad0498c9b48dd80220aeb46 P 5425fb70daaf4f5e8bf84f5e5143e89f: Bootstrap starting.
I20260806 09:32:00.630771 9374 tablet_bootstrap.cc:654] T 25ec81213ad0498c9b48dd80220aeb46 P 339f163711b143c6a6a210b47915fb59: Neither blocks nor log segments found. Creating new log.
I20260806 09:32:00.631394 9376 tablet_bootstrap.cc:654] T 25ec81213ad0498c9b48dd80220aeb46 P 7702d30841eb48b79a91eaaeaf45e426: Neither blocks nor log segments found. Creating new log.
I20260806 09:32:00.631938 9375 tablet_bootstrap.cc:654] T 25ec81213ad0498c9b48dd80220aeb46 P 5425fb70daaf4f5e8bf84f5e5143e89f: Neither blocks nor log segments found. Creating new log.
I20260806 09:32:00.632181 9374 log.cc:824] T 25ec81213ad0498c9b48dd80220aeb46 P 339f163711b143c6a6a210b47915fb59: Log is configured to *not* fsync() on all Append() calls
I20260806 09:32:00.632547 9376 log.cc:824] T 25ec81213ad0498c9b48dd80220aeb46 P 7702d30841eb48b79a91eaaeaf45e426: Log is configured to *not* fsync() on all Append() calls
I20260806 09:32:00.633239 9375 log.cc:824] T 25ec81213ad0498c9b48dd80220aeb46 P 5425fb70daaf4f5e8bf84f5e5143e89f: Log is configured to *not* fsync() on all Append() calls
I20260806 09:32:00.635572 9374 tablet_bootstrap.cc:492] T 25ec81213ad0498c9b48dd80220aeb46 P 339f163711b143c6a6a210b47915fb59: No bootstrap required, opened a new log
I20260806 09:32:00.635702 9376 tablet_bootstrap.cc:492] T 25ec81213ad0498c9b48dd80220aeb46 P 7702d30841eb48b79a91eaaeaf45e426: No bootstrap required, opened a new log
I20260806 09:32:00.635895 9374 ts_tablet_manager.cc:1397] T 25ec81213ad0498c9b48dd80220aeb46 P 339f163711b143c6a6a210b47915fb59: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.003s
I20260806 09:32:00.635982 9376 ts_tablet_manager.cc:1397] T 25ec81213ad0498c9b48dd80220aeb46 P 7702d30841eb48b79a91eaaeaf45e426: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.005s
I20260806 09:32:00.636624 9375 tablet_bootstrap.cc:492] T 25ec81213ad0498c9b48dd80220aeb46 P 5425fb70daaf4f5e8bf84f5e5143e89f: No bootstrap required, opened a new log
I20260806 09:32:00.636896 9375 ts_tablet_manager.cc:1397] T 25ec81213ad0498c9b48dd80220aeb46 P 5425fb70daaf4f5e8bf84f5e5143e89f: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.003s
I20260806 09:32:00.644132 9374 raft_consensus.cc:348] T 25ec81213ad0498c9b48dd80220aeb46 P 339f163711b143c6a6a210b47915fb59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "339f163711b143c6a6a210b47915fb59" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 39849 } } peers { permanent_uuid: "7702d30841eb48b79a91eaaeaf45e426" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 37455 } } peers { permanent_uuid: "5425fb70daaf4f5e8bf84f5e5143e89f" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 46039 } }
I20260806 09:32:00.644620 9374 raft_consensus.cc:374] T 25ec81213ad0498c9b48dd80220aeb46 P 339f163711b143c6a6a210b47915fb59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:32:00.644733 9374 raft_consensus.cc:729] T 25ec81213ad0498c9b48dd80220aeb46 P 339f163711b143c6a6a210b47915fb59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 339f163711b143c6a6a210b47915fb59, State: Initialized, Role: FOLLOWER
I20260806 09:32:00.645346 9374 consensus_queue.cc:260] T 25ec81213ad0498c9b48dd80220aeb46 P 339f163711b143c6a6a210b47915fb59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "339f163711b143c6a6a210b47915fb59" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 39849 } } peers { permanent_uuid: "7702d30841eb48b79a91eaaeaf45e426" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 37455 } } peers { permanent_uuid: "5425fb70daaf4f5e8bf84f5e5143e89f" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 46039 } }
I20260806 09:32:00.646477 9350 heartbeater.cc:503] Master 127.20.113.62:44265 was elected leader, sending a full tablet report...
I20260806 09:32:00.647240 9374 ts_tablet_manager.cc:1428] T 25ec81213ad0498c9b48dd80220aeb46 P 339f163711b143c6a6a210b47915fb59: Time spent starting tablet: real 0.011s user 0.008s sys 0.005s
I20260806 09:32:00.647617 9375 raft_consensus.cc:348] T 25ec81213ad0498c9b48dd80220aeb46 P 5425fb70daaf4f5e8bf84f5e5143e89f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "339f163711b143c6a6a210b47915fb59" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 39849 } } peers { permanent_uuid: "7702d30841eb48b79a91eaaeaf45e426" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 37455 } } peers { permanent_uuid: "5425fb70daaf4f5e8bf84f5e5143e89f" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 46039 } }
I20260806 09:32:00.648032 9375 raft_consensus.cc:374] T 25ec81213ad0498c9b48dd80220aeb46 P 5425fb70daaf4f5e8bf84f5e5143e89f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:32:00.648214 9375 raft_consensus.cc:729] T 25ec81213ad0498c9b48dd80220aeb46 P 5425fb70daaf4f5e8bf84f5e5143e89f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5425fb70daaf4f5e8bf84f5e5143e89f, State: Initialized, Role: FOLLOWER
I20260806 09:32:00.648353 9376 raft_consensus.cc:348] T 25ec81213ad0498c9b48dd80220aeb46 P 7702d30841eb48b79a91eaaeaf45e426 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "339f163711b143c6a6a210b47915fb59" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 39849 } } peers { permanent_uuid: "7702d30841eb48b79a91eaaeaf45e426" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 37455 } } peers { permanent_uuid: "5425fb70daaf4f5e8bf84f5e5143e89f" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 46039 } }
I20260806 09:32:00.648841 9376 raft_consensus.cc:374] T 25ec81213ad0498c9b48dd80220aeb46 P 7702d30841eb48b79a91eaaeaf45e426 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:32:00.648991 9376 raft_consensus.cc:729] T 25ec81213ad0498c9b48dd80220aeb46 P 7702d30841eb48b79a91eaaeaf45e426 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7702d30841eb48b79a91eaaeaf45e426, State: Initialized, Role: FOLLOWER
I20260806 09:32:00.649060 9375 consensus_queue.cc:260] T 25ec81213ad0498c9b48dd80220aeb46 P 5425fb70daaf4f5e8bf84f5e5143e89f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "339f163711b143c6a6a210b47915fb59" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 39849 } } peers { permanent_uuid: "7702d30841eb48b79a91eaaeaf45e426" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 37455 } } peers { permanent_uuid: "5425fb70daaf4f5e8bf84f5e5143e89f" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 46039 } }
I20260806 09:32:00.649636 9376 consensus_queue.cc:260] T 25ec81213ad0498c9b48dd80220aeb46 P 7702d30841eb48b79a91eaaeaf45e426 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "339f163711b143c6a6a210b47915fb59" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 39849 } } peers { permanent_uuid: "7702d30841eb48b79a91eaaeaf45e426" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 37455 } } peers { permanent_uuid: "5425fb70daaf4f5e8bf84f5e5143e89f" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 46039 } }
I20260806 09:32:00.651103 9375 ts_tablet_manager.cc:1428] T 25ec81213ad0498c9b48dd80220aeb46 P 5425fb70daaf4f5e8bf84f5e5143e89f: Time spent starting tablet: real 0.014s user 0.008s sys 0.005s
I20260806 09:32:00.651994 9152 heartbeater.cc:503] Master 127.20.113.62:44265 was elected leader, sending a full tablet report...
I20260806 09:32:00.652678 9376 ts_tablet_manager.cc:1428] T 25ec81213ad0498c9b48dd80220aeb46 P 7702d30841eb48b79a91eaaeaf45e426: Time spent starting tablet: real 0.017s user 0.013s sys 0.004s
W20260806 09:32:00.700112 9153 tablet.cc:2369] T 25ec81213ad0498c9b48dd80220aeb46 P 7702d30841eb48b79a91eaaeaf45e426: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 09:32:00.720352 9384 raft_consensus.cc:482] T 25ec81213ad0498c9b48dd80220aeb46 P 5425fb70daaf4f5e8bf84f5e5143e89f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 09:32:00.720649 9384 raft_consensus.cc:504] T 25ec81213ad0498c9b48dd80220aeb46 P 5425fb70daaf4f5e8bf84f5e5143e89f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "339f163711b143c6a6a210b47915fb59" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 39849 } } peers { permanent_uuid: "7702d30841eb48b79a91eaaeaf45e426" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 37455 } } peers { permanent_uuid: "5425fb70daaf4f5e8bf84f5e5143e89f" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 46039 } }
I20260806 09:32:00.722764 9384 leader_election.cc:288] T 25ec81213ad0498c9b48dd80220aeb46 P 5425fb70daaf4f5e8bf84f5e5143e89f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 339f163711b143c6a6a210b47915fb59 (127.20.113.3:39849), 7702d30841eb48b79a91eaaeaf45e426 (127.20.113.2:37455)
I20260806 09:32:00.729988 9371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:00.723323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.1:42021 (local address 127.20.113.2:37455)
0806 09:32:00.723553 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0806 09:32:00.723562 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 09:32:00.724500 (+ 938us) server_negotiation.cc:407] Connection header received
0806 09:32:00.724703 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:32:00.724708 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:32:00.724778 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:32:00.724873 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:32:00.725517 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:00.726221 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:00.728133 (+ 1912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:00.728994 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:00.729186 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:32:00.729271 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 09:32:00.729583 (+ 312us) server_negotiation.cc:299] Negotiation successful
0806 09:32:00.729710 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 09:32:00.730031 9368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:00.723215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.1:44079 (local address 127.20.113.3:39849)
0806 09:32:00.723453 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0806 09:32:00.723461 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 09:32:00.724349 (+ 888us) server_negotiation.cc:407] Connection header received
0806 09:32:00.724591 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:32:00.724595 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:32:00.724656 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:32:00.724757 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:32:00.725444 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:00.726210 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:00.728162 (+ 1952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:00.729020 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:00.729203 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:32:00.729287 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 09:32:00.729629 (+ 342us) server_negotiation.cc:299] Negotiation successful
0806 09:32:00.729755 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260806 09:32:00.730175 9389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:00.723355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.2:37455 (local address 127.20.113.1:42021)
0806 09:32:00.724365 (+ 1010us) negotiation.cc:107] Waiting for socket to connect
0806 09:32:00.724389 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 09:32:00.724554 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:32:00.724899 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:32:00.724904 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:32:00.724947 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:32:00.725347 (+ 400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:32:00.725356 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:32:00.726365 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:32:00.726371 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:32:00.727970 (+ 1599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:32:00.727985 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:32:00.729240 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:32:00.729247 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:32:00.729403 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:32:00.729432 (+ 29us) client_negotiation.cc:769] Sending connection context
0806 09:32:00.729555 (+ 123us) client_negotiation.cc:240] Negotiation successful
0806 09:32:00.729678 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":854,"mutex_wait_us":104,"thread_start_us":401,"threads_started":1}
I20260806 09:32:00.730772 9388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:00.723355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.3:39849 (local address 127.20.113.1:44079)
0806 09:32:00.724203 (+ 848us) negotiation.cc:107] Waiting for socket to connect
0806 09:32:00.724220 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 09:32:00.724415 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:32:00.724775 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:32:00.724780 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:32:00.724806 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:32:00.725270 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:32:00.725281 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:32:00.726355 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:32:00.726362 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:32:00.727998 (+ 1636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:32:00.728010 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:32:00.729287 (+ 1277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:32:00.729294 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:32:00.729450 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:32:00.729472 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 09:32:00.729598 (+ 126us) client_negotiation.cc:240] Negotiation successful
0806 09:32:00.729726 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":725,"mutex_wait_us":70,"thread_start_us":500,"threads_started":1}
I20260806 09:32:00.731365 9082 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25ec81213ad0498c9b48dd80220aeb46" candidate_uuid: "5425fb70daaf4f5e8bf84f5e5143e89f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7702d30841eb48b79a91eaaeaf45e426" is_pre_election: true
I20260806 09:32:00.731920 9082 raft_consensus.cc:2454] T 25ec81213ad0498c9b48dd80220aeb46 P 7702d30841eb48b79a91eaaeaf45e426 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5425fb70daaf4f5e8bf84f5e5143e89f in term 0.
I20260806 09:32:00.731848 9284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25ec81213ad0498c9b48dd80220aeb46" candidate_uuid: "5425fb70daaf4f5e8bf84f5e5143e89f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "339f163711b143c6a6a210b47915fb59" is_pre_election: true
I20260806 09:32:00.732486 9284 raft_consensus.cc:2454] T 25ec81213ad0498c9b48dd80220aeb46 P 339f163711b143c6a6a210b47915fb59 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5425fb70daaf4f5e8bf84f5e5143e89f in term 0.
I20260806 09:32:00.732681 8792 leader_election.cc:302] T 25ec81213ad0498c9b48dd80220aeb46 P 5425fb70daaf4f5e8bf84f5e5143e89f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5425fb70daaf4f5e8bf84f5e5143e89f, 7702d30841eb48b79a91eaaeaf45e426; no voters:
I20260806 09:32:00.733134 9384 raft_consensus.cc:2781] T 25ec81213ad0498c9b48dd80220aeb46 P 5425fb70daaf4f5e8bf84f5e5143e89f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 09:32:00.733325 9384 raft_consensus.cc:482] T 25ec81213ad0498c9b48dd80220aeb46 P 5425fb70daaf4f5e8bf84f5e5143e89f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 09:32:00.733436 9384 raft_consensus.cc:3037] T 25ec81213ad0498c9b48dd80220aeb46 P 5425fb70daaf4f5e8bf84f5e5143e89f [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:32:00.735894 9384 raft_consensus.cc:504] T 25ec81213ad0498c9b48dd80220aeb46 P 5425fb70daaf4f5e8bf84f5e5143e89f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "339f163711b143c6a6a210b47915fb59" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 39849 } } peers { permanent_uuid: "7702d30841eb48b79a91eaaeaf45e426" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 37455 } } peers { permanent_uuid: "5425fb70daaf4f5e8bf84f5e5143e89f" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 46039 } }
I20260806 09:32:00.736584 9384 leader_election.cc:288] T 25ec81213ad0498c9b48dd80220aeb46 P 5425fb70daaf4f5e8bf84f5e5143e89f [CANDIDATE]: Term 1 election: Requested vote from peers 339f163711b143c6a6a210b47915fb59 (127.20.113.3:39849), 7702d30841eb48b79a91eaaeaf45e426 (127.20.113.2:37455)
I20260806 09:32:00.736990 9284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25ec81213ad0498c9b48dd80220aeb46" candidate_uuid: "5425fb70daaf4f5e8bf84f5e5143e89f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "339f163711b143c6a6a210b47915fb59"
I20260806 09:32:00.737103 9082 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25ec81213ad0498c9b48dd80220aeb46" candidate_uuid: "5425fb70daaf4f5e8bf84f5e5143e89f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7702d30841eb48b79a91eaaeaf45e426"
I20260806 09:32:00.737185 9284 raft_consensus.cc:3037] T 25ec81213ad0498c9b48dd80220aeb46 P 339f163711b143c6a6a210b47915fb59 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:32:00.737296 9082 raft_consensus.cc:3037] T 25ec81213ad0498c9b48dd80220aeb46 P 7702d30841eb48b79a91eaaeaf45e426 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:32:00.739465 9284 raft_consensus.cc:2454] T 25ec81213ad0498c9b48dd80220aeb46 P 339f163711b143c6a6a210b47915fb59 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5425fb70daaf4f5e8bf84f5e5143e89f in term 1.
I20260806 09:32:00.739904 8797 leader_election.cc:302] T 25ec81213ad0498c9b48dd80220aeb46 P 5425fb70daaf4f5e8bf84f5e5143e89f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 339f163711b143c6a6a210b47915fb59, 5425fb70daaf4f5e8bf84f5e5143e89f; no voters:
I20260806 09:32:00.740237 9384 raft_consensus.cc:2781] T 25ec81213ad0498c9b48dd80220aeb46 P 5425fb70daaf4f5e8bf84f5e5143e89f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:32:00.740480 9082 raft_consensus.cc:2454] T 25ec81213ad0498c9b48dd80220aeb46 P 7702d30841eb48b79a91eaaeaf45e426 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5425fb70daaf4f5e8bf84f5e5143e89f in term 1.
I20260806 09:32:00.740844 9384 raft_consensus.cc:686] T 25ec81213ad0498c9b48dd80220aeb46 P 5425fb70daaf4f5e8bf84f5e5143e89f [term 1 LEADER]: Becoming Leader. State: Replica: 5425fb70daaf4f5e8bf84f5e5143e89f, State: Running, Role: LEADER
I20260806 09:32:00.741734 9384 consensus_queue.cc:237] T 25ec81213ad0498c9b48dd80220aeb46 P 5425fb70daaf4f5e8bf84f5e5143e89f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "339f163711b143c6a6a210b47915fb59" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 39849 } } peers { permanent_uuid: "7702d30841eb48b79a91eaaeaf45e426" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 37455 } } peers { permanent_uuid: "5425fb70daaf4f5e8bf84f5e5143e89f" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 46039 } }
I20260806 09:32:00.746464 8683 catalog_manager.cc:5495] T 25ec81213ad0498c9b48dd80220aeb46 P 5425fb70daaf4f5e8bf84f5e5143e89f reported cstate change: term changed from 0 to 1, leader changed from <none> to 5425fb70daaf4f5e8bf84f5e5143e89f (127.20.113.1). New cstate: current_term: 1 leader_uuid: "5425fb70daaf4f5e8bf84f5e5143e89f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "339f163711b143c6a6a210b47915fb59" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 39849 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7702d30841eb48b79a91eaaeaf45e426" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 37455 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5425fb70daaf4f5e8bf84f5e5143e89f" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 46039 } health_report { overall_health: HEALTHY } } }
W20260806 09:32:00.753399 9352 tablet.cc:2369] T 25ec81213ad0498c9b48dd80220aeb46 P 339f163711b143c6a6a210b47915fb59: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 09:32:00.764114 9373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:00.758774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44454 (local address 127.20.113.1:46039)
0806 09:32:00.759007 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0806 09:32:00.759011 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 09:32:00.759051 (+ 40us) server_negotiation.cc:407] Connection header received
0806 09:32:00.759248 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:32:00.759252 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:32:00.759348 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:32:00.759467 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 09:32:00.759820 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:00.760609 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:00.762101 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:00.762519 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:00.762615 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:32:00.763105 (+ 490us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 09:32:00.763577 (+ 472us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 09:32:00.763682 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 09:32:00.763768 (+ 86us) server_negotiation.cc:299] Negotiation successful
0806 09:32:00.763833 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260806 09:32:00.771966 9082 raft_consensus.cc:1261] T 25ec81213ad0498c9b48dd80220aeb46 P 7702d30841eb48b79a91eaaeaf45e426 [term 1 FOLLOWER]: Refusing update from remote peer 5425fb70daaf4f5e8bf84f5e5143e89f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 09:32:00.771983 9284 raft_consensus.cc:1261] T 25ec81213ad0498c9b48dd80220aeb46 P 339f163711b143c6a6a210b47915fb59 [term 1 FOLLOWER]: Refusing update from remote peer 5425fb70daaf4f5e8bf84f5e5143e89f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 09:32:00.772964 9390 consensus_queue.cc:1035] T 25ec81213ad0498c9b48dd80220aeb46 P 5425fb70daaf4f5e8bf84f5e5143e89f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7702d30841eb48b79a91eaaeaf45e426" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 37455 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 09:32:00.773368 9384 consensus_queue.cc:1035] T 25ec81213ad0498c9b48dd80220aeb46 P 5425fb70daaf4f5e8bf84f5e5143e89f [LEADER]: Connected to new peer: Peer: permanent_uuid: "339f163711b143c6a6a210b47915fb59" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 39849 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 09:32:00.784835 9394 mvcc.cc:205] Tried to move back new op lower bound from 7315491720272437248 to 7315491720162152448. Current Snapshot: MvccSnapshot[applied={T|T < 7315491720272437248}]
I20260806 09:32:00.786549 9395 mvcc.cc:205] Tried to move back new op lower bound from 7315491720272437248 to 7315491720162152448. Current Snapshot: MvccSnapshot[applied={T|T < 7315491720272437248}]
I20260806 09:32:00.799444 8685 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53168:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 09:32:00.799739 8685 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53168:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 09:32:00.803678 8685 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2ebb3c6330eb44f9b537d7c3007f0271: Sending DeleteTablet for 3 replicas of tablet 25ec81213ad0498c9b48dd80220aeb46
I20260806 09:32:00.804924 9249 tablet_service.cc:1514] Processing DeleteTablet for tablet 25ec81213ad0498c9b48dd80220aeb46 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:32:00 UTC) from {username='kudu', principal='kudu/127.20.113.62@KRBTEST.COM'} at 127.0.0.1:46132
I20260806 09:32:00.804993 8855 tablet_service.cc:1514] Processing DeleteTablet for tablet 25ec81213ad0498c9b48dd80220aeb46 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:32:00 UTC) from {username='kudu', principal='kudu/127.20.113.62@KRBTEST.COM'} at 127.0.0.1:44444
I20260806 09:32:00.805197 9050 tablet_service.cc:1514] Processing DeleteTablet for tablet 25ec81213ad0498c9b48dd80220aeb46 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:32:00 UTC) from {username='kudu', principal='kudu/127.20.113.62@KRBTEST.COM'} at 127.0.0.1:53736
I20260806 09:32:00.806238 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 8770
I20260806 09:32:00.817536 9413 tablet_replica.cc:330] T 25ec81213ad0498c9b48dd80220aeb46 P 7702d30841eb48b79a91eaaeaf45e426: stopping tablet replica
I20260806 09:32:00.818259 9413 raft_consensus.cc:2229] T 25ec81213ad0498c9b48dd80220aeb46 P 7702d30841eb48b79a91eaaeaf45e426 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 09:32:00.819011 9413 raft_consensus.cc:2258] T 25ec81213ad0498c9b48dd80220aeb46 P 7702d30841eb48b79a91eaaeaf45e426 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 09:32:00.821017 9410 tablet_replica.cc:330] T 25ec81213ad0498c9b48dd80220aeb46 P 339f163711b143c6a6a210b47915fb59: stopping tablet replica
I20260806 09:32:00.821623 9410 raft_consensus.cc:2229] T 25ec81213ad0498c9b48dd80220aeb46 P 339f163711b143c6a6a210b47915fb59 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 09:32:00.822355 9410 raft_consensus.cc:2258] T 25ec81213ad0498c9b48dd80220aeb46 P 339f163711b143c6a6a210b47915fb59 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 09:32:00.823009 9412 tablet_replica.cc:330] T 25ec81213ad0498c9b48dd80220aeb46 P 5425fb70daaf4f5e8bf84f5e5143e89f: stopping tablet replica
I20260806 09:32:00.823417 9412 raft_consensus.cc:2229] T 25ec81213ad0498c9b48dd80220aeb46 P 5425fb70daaf4f5e8bf84f5e5143e89f [term 1 LEADER]: Raft consensus shutting down.
I20260806 09:32:00.823485 9413 ts_tablet_manager.cc:1905] T 25ec81213ad0498c9b48dd80220aeb46 P 7702d30841eb48b79a91eaaeaf45e426: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 09:32:00.824117 9412 raft_consensus.cc:2258] T 25ec81213ad0498c9b48dd80220aeb46 P 5425fb70daaf4f5e8bf84f5e5143e89f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 09:32:00.826421 9410 ts_tablet_manager.cc:1905] T 25ec81213ad0498c9b48dd80220aeb46 P 339f163711b143c6a6a210b47915fb59: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 09:32:00.830076 9412 ts_tablet_manager.cc:1905] T 25ec81213ad0498c9b48dd80220aeb46 P 5425fb70daaf4f5e8bf84f5e5143e89f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 09:32:00.830732 9413 ts_tablet_manager.cc:1918] T 25ec81213ad0498c9b48dd80220aeb46 P 7702d30841eb48b79a91eaaeaf45e426: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 09:32:00.830960 9413 log.cc:1196] T 25ec81213ad0498c9b48dd80220aeb46 P 7702d30841eb48b79a91eaaeaf45e426: Deleting WAL directory at /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-1/wal/wals/25ec81213ad0498c9b48dd80220aeb46
I20260806 09:32:00.831615 9413 ts_tablet_manager.cc:1939] T 25ec81213ad0498c9b48dd80220aeb46 P 7702d30841eb48b79a91eaaeaf45e426: Deleting consensus metadata
Aug 06 09:32:00 dist-test-slave-45dx krb5kdc[8615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008718, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.113.1@KRBTEST.COM
I20260806 09:32:00.833689 8661 catalog_manager.cc:4841] TS 7702d30841eb48b79a91eaaeaf45e426 (127.20.113.2:37455): tablet 25ec81213ad0498c9b48dd80220aeb46 (table test-table [id=0aa5c908cf74404386a300ea42a3898a]) successfully deleted
I20260806 09:32:00.838030 9412 ts_tablet_manager.cc:1918] T 25ec81213ad0498c9b48dd80220aeb46 P 5425fb70daaf4f5e8bf84f5e5143e89f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 09:32:00.838305 9412 log.cc:1196] T 25ec81213ad0498c9b48dd80220aeb46 P 5425fb70daaf4f5e8bf84f5e5143e89f: Deleting WAL directory at /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-0/wal/wals/25ec81213ad0498c9b48dd80220aeb46
I20260806 09:32:00.838970 9412 ts_tablet_manager.cc:1939] T 25ec81213ad0498c9b48dd80220aeb46 P 5425fb70daaf4f5e8bf84f5e5143e89f: Deleting consensus metadata
I20260806 09:32:00.840965 8664 catalog_manager.cc:4841] TS 5425fb70daaf4f5e8bf84f5e5143e89f (127.20.113.1:46039): tablet 25ec81213ad0498c9b48dd80220aeb46 (table test-table [id=0aa5c908cf74404386a300ea42a3898a]) successfully deleted
I20260806 09:32:00.841774 9410 ts_tablet_manager.cc:1918] T 25ec81213ad0498c9b48dd80220aeb46 P 339f163711b143c6a6a210b47915fb59: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 09:32:00.842021 9410 log.cc:1196] T 25ec81213ad0498c9b48dd80220aeb46 P 339f163711b143c6a6a210b47915fb59: Deleting WAL directory at /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-2/wal/wals/25ec81213ad0498c9b48dd80220aeb46
I20260806 09:32:00.842511 9373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:00.807363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44460 (local address 127.20.113.1:46039)
0806 09:32:00.807540 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0806 09:32:00.807546 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:32:00.820160 (+ 12614us) server_negotiation.cc:407] Connection header received
0806 09:32:00.820450 (+ 290us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:32:00.820457 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:32:00.820561 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:32:00.820680 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:32:00.822843 (+ 2163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:00.824072 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:00.826730 (+ 2658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:00.827489 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:00.827605 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:32:00.833774 (+ 6169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:32:00.833790 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:32:00.833818 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:32:00.833880 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:32:00.838365 (+ 4485us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:32:00.839414 (+ 1049us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:32:00.839425 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:32:00.839428 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:32:00.839632 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:32:00.840196 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:32:00.840201 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:32:00.840205 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:32:00.840747 (+ 542us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:32:00.841143 (+ 396us) server_negotiation.cc:1035] Waiting for connection context
0806 09:32:00.841871 (+ 728us) server_negotiation.cc:299] Negotiation successful
0806 09:32:00.842135 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 09:32:00.843039 9410 ts_tablet_manager.cc:1939] T 25ec81213ad0498c9b48dd80220aeb46 P 339f163711b143c6a6a210b47915fb59: Deleting consensus metadata
I20260806 09:32:00.843237 8942 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:32:00.845037 8666 catalog_manager.cc:4841] TS 339f163711b143c6a6a210b47915fb59 (127.20.113.3:39849): tablet 25ec81213ad0498c9b48dd80220aeb46 (table test-table [id=0aa5c908cf74404386a300ea42a3898a]) successfully deleted
I20260806 09:32:00.942641 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 8770
I20260806 09:32:00.974643 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 8956
Aug 06 09:32:00 dist-test-slave-45dx krb5kdc[8615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008718, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.113.2@KRBTEST.COM
I20260806 09:32:00.990392 9371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:00.975559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53742 (local address 127.20.113.2:37455)
0806 09:32:00.975752 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0806 09:32:00.975756 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 09:32:00.975775 (+ 19us) server_negotiation.cc:407] Connection header received
0806 09:32:00.975873 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:32:00.975877 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:32:00.976012 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:32:00.976096 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:32:00.977519 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:00.978372 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:00.979670 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:00.980090 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:00.980183 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:32:00.984941 (+ 4758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:32:00.984953 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:32:00.984969 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:32:00.985003 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:32:00.987738 (+ 2735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:32:00.988482 (+ 744us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:32:00.988489 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:32:00.988491 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:32:00.988623 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:32:00.988967 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:32:00.988970 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:32:00.988972 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:32:00.989344 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:32:00.989565 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0806 09:32:00.989837 (+ 272us) server_negotiation.cc:299] Negotiation successful
0806 09:32:00.990012 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260806 09:32:00.991122 9146 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:32:01.082494 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 8956
I20260806 09:32:01.109048 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 9158
Aug 06 09:32:01 dist-test-slave-45dx krb5kdc[8615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008718, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.113.3@KRBTEST.COM
I20260806 09:32:01.124485 9368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:01.109862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46148 (local address 127.20.113.3:39849)
0806 09:32:01.110067 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0806 09:32:01.110072 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:32:01.110093 (+ 21us) server_negotiation.cc:407] Connection header received
0806 09:32:01.110278 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:32:01.110282 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:32:01.110362 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:32:01.110468 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:32:01.111833 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:01.112616 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:01.113896 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:01.114308 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:01.114442 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:32:01.119040 (+ 4598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:32:01.119056 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:32:01.119073 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:32:01.119106 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:32:01.121855 (+ 2749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:32:01.122493 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:32:01.122500 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:32:01.122502 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:32:01.122662 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:32:01.123002 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:32:01.123005 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:32:01.123007 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:32:01.123326 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:32:01.123548 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0806 09:32:01.123816 (+ 268us) server_negotiation.cc:299] Negotiation successful
0806 09:32:01.123995 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 09:32:01.125108 9342 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:32:01.213181 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 9158
I20260806 09:32:01.239331 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 8634
I20260806 09:32:01.240814 8726 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:32:01.353960 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 8634
I20260806 09:32:01.385607 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.113.62:44265
--webserver_interface=127.20.113.62
--webserver_port=34199
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab
--principal=kudu/127.20.113.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:46541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.113.62:44265
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:32:01.735698 9423 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:32:01.736060 9423 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:32:01.736219 9423 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:32:01.743564 9423 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 09:32:01.743714 9423 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:32:01.743818 9423 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:32:01.743927 9423 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 09:32:01.744004 9423 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 09:32:01.753844 9423 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:46541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.20.113.62:44265
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab
--principal=kudu/127.20.113.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.113.62:44265
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.113.62
--webserver_port=34199
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:32:01.755159 9423 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:32:01.756501 9423 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:32:01.784123 9440 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:32:01.784260 9437 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:32:01.784287 9434 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:32:01.785193 9423 server_base.cc:1034] running on GCE node
I20260806 09:32:01.786082 9423 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:32:01.787690 9423 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:32:01.788959 9423 hybrid_clock.cc:648] HybridClock initialized: now 1786008721788898 us; error 57 us; skew 500 ppm
I20260806 09:32:01.789377 9423 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:32:01 dist-test-slave-45dx krb5kdc[8615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008721, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:32:01.798081 9423 init.cc:377] Logged in from keytab as kudu/127.20.113.62@KRBTEST.COM (short username kudu)
I20260806 09:32:01.799835 9423 webserver.cc:460] Webserver started at http://127.20.113.62:34199/ using document root <none> and password file <none>
I20260806 09:32:01.800689 9423 fs_manager.cc:359] Metadata directory not provided
I20260806 09:32:01.800869 9423 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:32:01.807399 9423 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 09:32:01.810431 9448 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:32:01.811740 9423 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 09:32:01.811949 9423 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/master-0/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/master-0/wal
uuid: "2ebb3c6330eb44f9b537d7c3007f0271"
format_stamp: "Formatted at 2026-08-06 09:31:58 on dist-test-slave-45dx"
server_key: "05ba32b4bb156010031014cc0f150aea"
server_key_iv: "9dbb2824c88d4631079ed6f5d6310224"
server_key_version: "encryptionkey@0"
I20260806 09:32:01.813122 9423 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:32:01.836632 9423 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:32:01.841423 9423 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:32:01.841769 9423 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:32:01.861456 9423 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.62:44265
I20260806 09:32:01.861476 9528 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.62:44265 every 8 connection(s)
I20260806 09:32:01.863065 9423 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
I20260806 09:32:01.867779 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 9423
I20260806 09:32:01.868851 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.113.1:46039
--local_ip_for_outbound_sockets=127.20.113.1
--tserver_master_addrs=127.20.113.62:44265
--webserver_port=45327
--webserver_interface=127.20.113.1
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.1.keytab
--principal=kudu/127.20.113.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:46541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 09:32:01.871460 9530 sys_catalog.cc:263] Verifying existing consensus state
I20260806 09:32:01.876744 9530 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ebb3c6330eb44f9b537d7c3007f0271: Bootstrap starting.
I20260806 09:32:01.898336 9530 log.cc:824] T 00000000000000000000000000000000 P 2ebb3c6330eb44f9b537d7c3007f0271: Log is configured to *not* fsync() on all Append() calls
I20260806 09:32:01.911747 9530 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ebb3c6330eb44f9b537d7c3007f0271: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 09:32:01.912545 9530 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ebb3c6330eb44f9b537d7c3007f0271: Bootstrap complete.
I20260806 09:32:01.926504 9530 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2ebb3c6330eb44f9b537d7c3007f0271 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ebb3c6330eb44f9b537d7c3007f0271" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 44265 } }
I20260806 09:32:01.928013 9530 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2ebb3c6330eb44f9b537d7c3007f0271 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ebb3c6330eb44f9b537d7c3007f0271, State: Initialized, Role: FOLLOWER
I20260806 09:32:01.928843 9530 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2ebb3c6330eb44f9b537d7c3007f0271 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ebb3c6330eb44f9b537d7c3007f0271" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 44265 } }
I20260806 09:32:01.929176 9530 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2ebb3c6330eb44f9b537d7c3007f0271 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 09:32:01.929311 9530 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2ebb3c6330eb44f9b537d7c3007f0271 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 09:32:01.929522 9530 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2ebb3c6330eb44f9b537d7c3007f0271 [term 1 FOLLOWER]: Advancing to term 2
I20260806 09:32:01.935986 9530 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2ebb3c6330eb44f9b537d7c3007f0271 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ebb3c6330eb44f9b537d7c3007f0271" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 44265 } }
I20260806 09:32:01.936740 9530 leader_election.cc:302] T 00000000000000000000000000000000 P 2ebb3c6330eb44f9b537d7c3007f0271 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2ebb3c6330eb44f9b537d7c3007f0271; no voters:
I20260806 09:32:01.937464 9530 leader_election.cc:288] T 00000000000000000000000000000000 P 2ebb3c6330eb44f9b537d7c3007f0271 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 09:32:01.937706 9536 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2ebb3c6330eb44f9b537d7c3007f0271 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 09:32:01.939759 9530 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2ebb3c6330eb44f9b537d7c3007f0271 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 09:32:01.940972 9536 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2ebb3c6330eb44f9b537d7c3007f0271 [term 2 LEADER]: Becoming Leader. State: Replica: 2ebb3c6330eb44f9b537d7c3007f0271, State: Running, Role: LEADER
I20260806 09:32:01.941932 9536 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2ebb3c6330eb44f9b537d7c3007f0271 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ebb3c6330eb44f9b537d7c3007f0271" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 44265 } }
I20260806 09:32:01.944993 9538 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ebb3c6330eb44f9b537d7c3007f0271 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "2ebb3c6330eb44f9b537d7c3007f0271" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ebb3c6330eb44f9b537d7c3007f0271" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 44265 } } }
I20260806 09:32:01.945017 9539 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ebb3c6330eb44f9b537d7c3007f0271 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2ebb3c6330eb44f9b537d7c3007f0271. Latest consensus state: current_term: 2 leader_uuid: "2ebb3c6330eb44f9b537d7c3007f0271" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ebb3c6330eb44f9b537d7c3007f0271" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 44265 } } }
I20260806 09:32:01.945514 9538 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ebb3c6330eb44f9b537d7c3007f0271 [sys.catalog]: This master's current role is: LEADER
I20260806 09:32:01.945650 9539 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ebb3c6330eb44f9b537d7c3007f0271 [sys.catalog]: This master's current role is: LEADER
I20260806 09:32:01.956148 9546 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 09:32:01.967829 9546 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 09:32:01.969678 9546 catalog_manager.cc:1259] Loaded cluster ID: 77281e58ac894f2c99060d83083d2db4
I20260806 09:32:01.969807 9546 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 09:32:01.973943 9546 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260806 09:32:01.994761 9546 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260806 09:32:02.002202 9546 catalog_manager.cc:1504] Loading token signing keys...
I20260806 09:32:02.003528 9546 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 2ebb3c6330eb44f9b537d7c3007f0271: Loaded TSK: 0
I20260806 09:32:02.004736 9546 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 09:32:02.275579 9532 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:32:02.275903 9532 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:32:02.276077 9532 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:32:02.283411 9532 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:32:02.283565 9532 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:32:02.283743 9532 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.113.1
I20260806 09:32:02.293572 9532 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:46541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.1.keytab
--principal=kudu/127.20.113.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.113.1:46039
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.113.1
--webserver_port=45327
--tserver_master_addrs=127.20.113.62:44265
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.113.1
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:32:02.294857 9532 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:32:02.296151 9532 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:32:02.310401 9572 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:32:02.310500 9574 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:32:02.311194 9571 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:32:02.311497 9532 server_base.cc:1034] running on GCE node
I20260806 09:32:02.312100 9532 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:32:02.313230 9532 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:32:02.314469 9532 hybrid_clock.cc:648] HybridClock initialized: now 1786008722314411 us; error 50 us; skew 500 ppm
I20260806 09:32:02.314800 9532 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:32:02 dist-test-slave-45dx krb5kdc[8615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008722, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:32:02.318339 9532 init.cc:377] Logged in from keytab as kudu/127.20.113.1@KRBTEST.COM (short username kudu)
I20260806 09:32:02.319948 9532 webserver.cc:460] Webserver started at http://127.20.113.1:45327/ using document root <none> and password file <none>
I20260806 09:32:02.320694 9532 fs_manager.cc:359] Metadata directory not provided
I20260806 09:32:02.320852 9532 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:32:02.326399 9532 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.003s
I20260806 09:32:02.329312 9584 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:32:02.330829 9532 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 09:32:02.331014 9532 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-0/wal
uuid: "5425fb70daaf4f5e8bf84f5e5143e89f"
format_stamp: "Formatted at 2026-08-06 09:31:59 on dist-test-slave-45dx"
server_key: "29d545bab1cc17611d12fb3650b096a5"
server_key_iv: "96d0310e72ed5b248b2fbe011b5f80c5"
server_key_version: "encryptionkey@0"
I20260806 09:32:02.332063 9532 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:32:02.351137 9532 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:32:02.355794 9532 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:32:02.356089 9532 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:32:02.357170 9532 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:32:02.358580 9532 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:32:02.358698 9532 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:32:02.358841 9532 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:32:02.358953 9532 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:32:02.393707 9532 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.1:46039
I20260806 09:32:02.393797 9752 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.1:46039 every 8 connection(s)
I20260806 09:32:02.395092 9532 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-0/data/info.pb
I20260806 09:32:02.404788 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 9532
I20260806 09:32:02.405537 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.113.2:37455
--local_ip_for_outbound_sockets=127.20.113.2
--tserver_master_addrs=127.20.113.62:44265
--webserver_port=45355
--webserver_interface=127.20.113.2
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.2.keytab
--principal=kudu/127.20.113.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:46541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 09:32:02 dist-test-slave-45dx krb5kdc[8615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008722, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.1@KRBTEST.COM for kudu/127.20.113.62@KRBTEST.COM
I20260806 09:32:02.419962 9760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:02.398092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44265 (local address 127.20.113.1:47523)
0806 09:32:02.398991 (+ 899us) negotiation.cc:107] Waiting for socket to connect
0806 09:32:02.399047 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0806 09:32:02.400919 (+ 1872us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:32:02.402235 (+ 1316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:32:02.402243 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:32:02.402736 (+ 493us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:32:02.403283 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:32:02.403299 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:32:02.405270 (+ 1971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:32:02.405274 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:32:02.406524 (+ 1250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:32:02.406536 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:32:02.408003 (+ 1467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:32:02.408009 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:32:02.408193 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:32:02.408959 (+ 766us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:32:02.408980 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:32:02.411148 (+ 2168us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:32:02.415837 (+ 4689us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:32:02.415843 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:32:02.415853 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:32:02.416276 (+ 423us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:32:02.416818 (+ 542us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:32:02.416821 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:32:02.416823 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:32:02.417062 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:32:02.418020 (+ 958us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:32:02.418025 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:32:02.418311 (+ 286us) client_negotiation.cc:769] Sending connection context
0806 09:32:02.418547 (+ 236us) client_negotiation.cc:240] Negotiation successful
0806 09:32:02.418930 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":599,"thread_start_us":416,"threads_started":1}
I20260806 09:32:02.420907 9756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:02.398048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.1:47523 (local address 127.20.113.62:44265)
0806 09:32:02.398729 (+ 681us) server_negotiation.cc:206] Beginning negotiation
0806 09:32:02.398742 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 09:32:02.399178 (+ 436us) server_negotiation.cc:407] Connection header received
0806 09:32:02.401417 (+ 2239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:32:02.401430 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:32:02.401724 (+ 294us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:32:02.402069 (+ 345us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:32:02.403482 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:02.405115 (+ 1633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:02.406693 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:02.407768 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:02.407956 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:32:02.411429 (+ 3473us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:32:02.411441 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:32:02.411463 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:32:02.411512 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:32:02.415621 (+ 4109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:32:02.416450 (+ 829us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:32:02.416461 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:32:02.416470 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:32:02.416699 (+ 229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:32:02.417191 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:32:02.417198 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:32:02.417203 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:32:02.417881 (+ 678us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:32:02.418078 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0806 09:32:02.418818 (+ 740us) server_negotiation.cc:299] Negotiation successful
0806 09:32:02.419266 (+ 448us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":485,"thread_start_us":336,"threads_started":1}
I20260806 09:32:02.423727 9753 heartbeater.cc:347] Connected to a master server at 127.20.113.62:44265
I20260806 09:32:02.424059 9753 heartbeater.cc:464] Registering TS with master...
I20260806 09:32:02.424896 9753 heartbeater.cc:511] Master 127.20.113.62:44265 requested a full tablet report, sending...
I20260806 09:32:02.427192 9469 ts_manager.cc:194] Registered new tserver with Master: 5425fb70daaf4f5e8bf84f5e5143e89f (127.20.113.1:46039)
I20260806 09:32:02.428817 9469 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.113.1@KRBTEST.COM'} at 127.20.113.1:47523
W20260806 09:32:02.760667 9761 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:32:02.760969 9761 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:32:02.761147 9761 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:32:02.768313 9761 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:32:02.768450 9761 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:32:02.768635 9761 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.113.2
I20260806 09:32:02.777797 9761 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:46541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.2.keytab
--principal=kudu/127.20.113.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.113.2:37455
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.113.2
--webserver_port=45355
--tserver_master_addrs=127.20.113.62:44265
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.113.2
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:32:02.779053 9761 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:32:02.780423 9761 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:32:02.796525 9767 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:32:02.796628 9769 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:32:02.796838 9773 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:32:02.797400 9761 server_base.cc:1034] running on GCE node
I20260806 09:32:02.797993 9761 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:32:02.798930 9761 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:32:02.800179 9761 hybrid_clock.cc:648] HybridClock initialized: now 1786008722800132 us; error 31 us; skew 500 ppm
I20260806 09:32:02.800525 9761 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:32:02 dist-test-slave-45dx krb5kdc[8615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008722, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:32:02.803653 9761 init.cc:377] Logged in from keytab as kudu/127.20.113.2@KRBTEST.COM (short username kudu)
I20260806 09:32:02.805088 9761 webserver.cc:460] Webserver started at http://127.20.113.2:45355/ using document root <none> and password file <none>
I20260806 09:32:02.805749 9761 fs_manager.cc:359] Metadata directory not provided
I20260806 09:32:02.805917 9761 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:32:02.811247 9761 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 09:32:02.813964 9782 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:32:02.815218 9761 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260806 09:32:02.815412 9761 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-1/wal
uuid: "7702d30841eb48b79a91eaaeaf45e426"
format_stamp: "Formatted at 2026-08-06 09:31:59 on dist-test-slave-45dx"
server_key: "b4b9ffe9ba6a109c233ab2bf8d6dc65e"
server_key_iv: "31395436d915c43eeceff885ef75b388"
server_key_version: "encryptionkey@0"
I20260806 09:32:02.816434 9761 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:32:02.841385 9761 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:32:02.845417 9761 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:32:02.845707 9761 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:32:02.846678 9761 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:32:02.847970 9761 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:32:02.848094 9761 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:32:02.848229 9761 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:32:02.848366 9761 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:32:02.884205 9761 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.2:37455
I20260806 09:32:02.884339 9933 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.2:37455 every 8 connection(s)
I20260806 09:32:02.885627 9761 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-1/data/info.pb
I20260806 09:32:02.890844 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 9761
I20260806 09:32:02.891458 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.113.3:39849
--local_ip_for_outbound_sockets=127.20.113.3
--tserver_master_addrs=127.20.113.62:44265
--webserver_port=34351
--webserver_interface=127.20.113.3
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.3.keytab
--principal=kudu/127.20.113.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:46541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 09:32:02 dist-test-slave-45dx krb5kdc[8615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008722, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.2@KRBTEST.COM for kudu/127.20.113.62@KRBTEST.COM
I20260806 09:32:02.905514 9756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:02.888174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.2:35719 (local address 127.20.113.62:44265)
0806 09:32:02.888433 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0806 09:32:02.888438 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:32:02.889582 (+ 1144us) server_negotiation.cc:407] Connection header received
0806 09:32:02.891526 (+ 1944us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:32:02.891531 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:32:02.891652 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:32:02.891778 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:32:02.893315 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:02.893998 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:02.895526 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:02.895984 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:02.896121 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:32:02.899812 (+ 3691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:32:02.899817 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:32:02.899820 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:32:02.899850 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:32:02.902485 (+ 2635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:32:02.903212 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:32:02.903217 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:32:02.903220 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:32:02.903356 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:32:02.903804 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:32:02.903808 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:32:02.903810 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:32:02.904090 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:32:02.904261 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0806 09:32:02.904942 (+ 681us) server_negotiation.cc:299] Negotiation successful
0806 09:32:02.905169 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260806 09:32:02.906416 9938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:02.888697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44265 (local address 127.20.113.2:35719)
0806 09:32:02.889420 (+ 723us) negotiation.cc:107] Waiting for socket to connect
0806 09:32:02.889465 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 09:32:02.891243 (+ 1778us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:32:02.892012 (+ 769us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:32:02.892021 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:32:02.892604 (+ 583us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:32:02.893140 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:32:02.893157 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:32:02.894170 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:32:02.894174 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:32:02.895370 (+ 1196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:32:02.895382 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:32:02.896213 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:32:02.896221 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:32:02.896504 (+ 283us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:32:02.897192 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:32:02.897214 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:32:02.899585 (+ 2371us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:32:02.902632 (+ 3047us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:32:02.902641 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:32:02.902655 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:32:02.903075 (+ 420us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:32:02.903470 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:32:02.903474 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:32:02.903478 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:32:02.903669 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:32:02.904227 (+ 558us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:32:02.904235 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:32:02.904574 (+ 339us) client_negotiation.cc:769] Sending connection context
0806 09:32:02.904828 (+ 254us) client_negotiation.cc:240] Negotiation successful
0806 09:32:02.905335 (+ 507us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":298,"threads_started":1}
I20260806 09:32:02.907896 9935 heartbeater.cc:347] Connected to a master server at 127.20.113.62:44265
I20260806 09:32:02.908191 9935 heartbeater.cc:464] Registering TS with master...
I20260806 09:32:02.908965 9935 heartbeater.cc:511] Master 127.20.113.62:44265 requested a full tablet report, sending...
I20260806 09:32:02.910498 9469 ts_manager.cc:194] Registered new tserver with Master: 7702d30841eb48b79a91eaaeaf45e426 (127.20.113.2:37455)
I20260806 09:32:02.911320 9469 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.113.2@KRBTEST.COM'} at 127.20.113.2:35719
W20260806 09:32:03.240963 9939 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:32:03.241247 9939 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:32:03.241428 9939 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:32:03.248548 9939 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:32:03.248696 9939 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:32:03.248871 9939 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.113.3
I20260806 09:32:03.258404 9939 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:46541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.3.keytab
--principal=kudu/127.20.113.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.113.3:39849
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.113.3
--webserver_port=34351
--tserver_master_addrs=127.20.113.62:44265
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.113.3
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:32:03.259649 9939 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:32:03.260989 9939 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:32:03.274216 9946 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:32:03.274240 9949 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:32:03.274178 9947 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:32:03.275393 9939 server_base.cc:1034] running on GCE node
I20260806 09:32:03.275981 9939 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:32:03.276971 9939 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:32:03.278183 9939 hybrid_clock.cc:648] HybridClock initialized: now 1786008723278148 us; error 28 us; skew 500 ppm
I20260806 09:32:03.278518 9939 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:32:03 dist-test-slave-45dx krb5kdc[8615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008723, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:32:03.281886 9939 init.cc:377] Logged in from keytab as kudu/127.20.113.3@KRBTEST.COM (short username kudu)
I20260806 09:32:03.283383 9939 webserver.cc:460] Webserver started at http://127.20.113.3:34351/ using document root <none> and password file <none>
I20260806 09:32:03.284027 9939 fs_manager.cc:359] Metadata directory not provided
I20260806 09:32:03.284188 9939 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:32:03.289562 9939 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 09:32:03.292227 9959 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:32:03.293514 9939 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 09:32:03.293686 9939 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-2/wal
uuid: "339f163711b143c6a6a210b47915fb59"
format_stamp: "Formatted at 2026-08-06 09:32:00 on dist-test-slave-45dx"
server_key: "e8f9c570bff6a1289f44f97f65712e3a"
server_key_iv: "706b241d6e9475c962eac2b2696c4650"
server_key_version: "encryptionkey@0"
I20260806 09:32:03.294595 9939 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:32:03.306489 9939 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:32:03.310629 9939 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:32:03.310905 9939 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:32:03.311838 9939 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:32:03.313102 9939 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:32:03.313233 9939 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:32:03.313371 9939 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:32:03.313462 9939 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:32:03.345922 9939 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.3:39849
I20260806 09:32:03.345980 10129 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.3:39849 every 8 connection(s)
I20260806 09:32:03.347431 9939 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-2/data/info.pb
I20260806 09:32:03.355067 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 9939
Aug 06 09:32:03 dist-test-slave-45dx krb5kdc[8615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008723, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.3@KRBTEST.COM for kudu/127.20.113.62@KRBTEST.COM
I20260806 09:32:03.369419 9756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:03.350051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.3:43859 (local address 127.20.113.62:44265)
0806 09:32:03.350234 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0806 09:32:03.350238 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 09:32:03.351371 (+ 1133us) server_negotiation.cc:407] Connection header received
0806 09:32:03.353292 (+ 1921us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:32:03.353298 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:32:03.353383 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:32:03.353498 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:32:03.354943 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:03.355799 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:03.358792 (+ 2993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:03.359245 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:03.359339 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:32:03.363135 (+ 3796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:32:03.363139 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:32:03.363143 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:32:03.363171 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:32:03.365909 (+ 2738us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:32:03.366852 (+ 943us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:32:03.366855 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:32:03.366857 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:32:03.366972 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:32:03.367487 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:32:03.367490 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:32:03.367492 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:32:03.367721 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:32:03.367825 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0806 09:32:03.368795 (+ 970us) server_negotiation.cc:299] Negotiation successful
0806 09:32:03.369036 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260806 09:32:03.370219 10135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:03.350533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44265 (local address 127.20.113.3:43859)
0806 09:32:03.351209 (+ 676us) negotiation.cc:107] Waiting for socket to connect
0806 09:32:03.351253 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 09:32:03.353025 (+ 1772us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:32:03.353733 (+ 708us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:32:03.353741 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:32:03.354245 (+ 504us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:32:03.354767 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:32:03.354783 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:32:03.355994 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:32:03.356002 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:32:03.358619 (+ 2617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:32:03.358641 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:32:03.359480 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:32:03.359492 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:32:03.359808 (+ 316us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:32:03.360565 (+ 757us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:32:03.360587 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:32:03.362948 (+ 2361us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:32:03.366046 (+ 3098us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:32:03.366058 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:32:03.366076 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:32:03.366734 (+ 658us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:32:03.367084 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:32:03.367090 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:32:03.367095 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:32:03.367372 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:32:03.367861 (+ 489us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:32:03.367871 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:32:03.368267 (+ 396us) client_negotiation.cc:769] Sending connection context
0806 09:32:03.368620 (+ 353us) client_negotiation.cc:240] Negotiation successful
0806 09:32:03.369012 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":265,"threads_started":1}
I20260806 09:32:03.371670 10131 heartbeater.cc:347] Connected to a master server at 127.20.113.62:44265
I20260806 09:32:03.372016 10131 heartbeater.cc:464] Registering TS with master...
I20260806 09:32:03.372844 10131 heartbeater.cc:511] Master 127.20.113.62:44265 requested a full tablet report, sending...
I20260806 09:32:03.374516 9469 ts_manager.cc:194] Registered new tserver with Master: 339f163711b143c6a6a210b47915fb59 (127.20.113.3:39849)
I20260806 09:32:03.374660 10137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:03.356101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44704 (local address 127.20.113.62:44265)
0806 09:32:03.359257 (+ 3156us) server_negotiation.cc:206] Beginning negotiation
0806 09:32:03.359261 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 09:32:03.359276 (+ 15us) server_negotiation.cc:407] Connection header received
0806 09:32:03.359349 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:32:03.359354 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:32:03.359430 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:32:03.359549 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:32:03.361153 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:03.362150 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:03.364476 (+ 2326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:03.365114 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:03.365271 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:32:03.368563 (+ 3292us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:32:03.368567 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:32:03.368570 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:32:03.368617 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:32:03.371944 (+ 3327us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:32:03.372730 (+ 786us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:32:03.372735 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:32:03.372738 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:32:03.372881 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:32:03.373301 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:32:03.373306 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:32:03.373309 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:32:03.373656 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:32:03.373829 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0806 09:32:03.374207 (+ 378us) server_negotiation.cc:299] Negotiation successful
0806 09:32:03.374354 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3070,"thread_start_us":2907,"threads_started":1}
I20260806 09:32:03.375599 9469 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.113.3@KRBTEST.COM'} at 127.20.113.3:43859
I20260806 09:32:03.377362 20932 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 09:32:03.384019 10137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:03.378313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44718 (local address 127.20.113.62:44265)
0806 09:32:03.378553 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0806 09:32:03.378561 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 09:32:03.378892 (+ 331us) server_negotiation.cc:407] Connection header received
0806 09:32:03.379040 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:32:03.379044 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:32:03.379155 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:32:03.379264 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 09:32:03.379654 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:03.380279 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:03.381971 (+ 1692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:03.382373 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:03.382505 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:32:03.382999 (+ 494us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 09:32:03.383485 (+ 486us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 09:32:03.383636 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0806 09:32:03.383713 (+ 77us) server_negotiation.cc:299] Negotiation successful
0806 09:32:03.383785 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 09:32:03.384864 9469 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:44718:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 09:32:03.391886 9469 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 09:32:03.431804 9753 heartbeater.cc:503] Master 127.20.113.62:44265 was elected leader, sending a full tablet report...
I20260806 09:32:03.437258 10147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:03.404193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.3:39849 (local address 127.0.0.1:44670)
0806 09:32:03.428767 (+ 24574us) negotiation.cc:107] Waiting for socket to connect
0806 09:32:03.428786 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 09:32:03.428905 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:32:03.429958 (+ 1053us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:32:03.429963 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:32:03.429992 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:32:03.430423 (+ 431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:32:03.430435 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:32:03.432100 (+ 1665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:32:03.432108 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:32:03.434403 (+ 2295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:32:03.434420 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:32:03.435942 (+ 1522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:32:03.435948 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:32:03.436158 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:32:03.436194 (+ 36us) client_negotiation.cc:769] Sending connection context
0806 09:32:03.436469 (+ 275us) client_negotiation.cc:240] Negotiation successful
0806 09:32:03.436661 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24426,"mutex_wait_us":85,"thread_start_us":23505,"threads_started":1}
I20260806 09:32:03.437605 10152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:03.404945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44670 (local address 127.20.113.3:39849)
0806 09:32:03.425289 (+ 20344us) server_negotiation.cc:206] Beginning negotiation
0806 09:32:03.425303 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 09:32:03.429083 (+ 3780us) server_negotiation.cc:407] Connection header received
0806 09:32:03.429202 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:32:03.429211 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:32:03.429472 (+ 261us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:32:03.429667 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:32:03.430637 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:03.431926 (+ 1289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:03.434635 (+ 2709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:03.435669 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:03.435823 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:32:03.436071 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0806 09:32:03.436518 (+ 447us) server_negotiation.cc:299] Negotiation successful
0806 09:32:03.436681 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20077,"thread_start_us":20370,"threads_started":1}
I20260806 09:32:03.438179 10145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:03.403643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.1:46039 (local address 127.0.0.1:44700)
0806 09:32:03.427452 (+ 23809us) negotiation.cc:107] Waiting for socket to connect
0806 09:32:03.427474 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 09:32:03.427602 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:32:03.428392 (+ 790us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:32:03.428401 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:32:03.428468 (+ 67us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:32:03.429044 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:32:03.429061 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:32:03.431121 (+ 2060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:32:03.431128 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:32:03.434635 (+ 3507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:32:03.434649 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:32:03.436799 (+ 2150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:32:03.436805 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:32:03.437076 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:32:03.437114 (+ 38us) client_negotiation.cc:769] Sending connection context
0806 09:32:03.437200 (+ 86us) client_negotiation.cc:240] Negotiation successful
0806 09:32:03.437346 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23638,"mutex_wait_us":282,"thread_start_us":26112,"threads_started":1}
I20260806 09:32:03.438488 10151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:03.404400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44700 (local address 127.20.113.1:46039)
0806 09:32:03.427737 (+ 23337us) server_negotiation.cc:206] Beginning negotiation
0806 09:32:03.427746 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 09:32:03.427778 (+ 32us) server_negotiation.cc:407] Connection header received
0806 09:32:03.427881 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:32:03.427895 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:32:03.428087 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:32:03.428340 (+ 253us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:32:03.429271 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:03.430970 (+ 1699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:03.434805 (+ 3835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:03.436262 (+ 1457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:03.436413 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:32:03.436624 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0806 09:32:03.437946 (+ 1322us) server_negotiation.cc:299] Negotiation successful
0806 09:32:03.438125 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23154,"thread_start_us":23517,"threads_started":1}
I20260806 09:32:03.438602 10146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:03.403714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58458 (local address 127.20.113.2:37455)
0806 09:32:03.429016 (+ 25302us) server_negotiation.cc:206] Beginning negotiation
0806 09:32:03.429024 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 09:32:03.429069 (+ 45us) server_negotiation.cc:407] Connection header received
0806 09:32:03.429172 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:32:03.429182 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:32:03.429389 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:32:03.429574 (+ 185us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:32:03.430239 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:03.431762 (+ 1523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:03.434483 (+ 2721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:03.435879 (+ 1396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:03.436022 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:32:03.436204 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0806 09:32:03.437485 (+ 1281us) server_negotiation.cc:299] Negotiation successful
0806 09:32:03.437649 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25120,"thread_start_us":25013,"threads_started":1}
I20260806 09:32:03.437348 10150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:03.403436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.2:37455 (local address 127.0.0.1:58458)
0806 09:32:03.426339 (+ 22903us) negotiation.cc:107] Waiting for socket to connect
0806 09:32:03.426420 (+ 81us) client_negotiation.cc:174] Beginning negotiation
0806 09:32:03.426688 (+ 268us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:32:03.429565 (+ 2877us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:32:03.429569 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:32:03.429595 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:32:03.430024 (+ 429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:32:03.430039 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:32:03.431951 (+ 1912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:32:03.431956 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:32:03.434309 (+ 2353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:32:03.434326 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:32:03.436387 (+ 2061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:32:03.436394 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:32:03.436591 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:32:03.436627 (+ 36us) client_negotiation.cc:769] Sending connection context
0806 09:32:03.436721 (+ 94us) client_negotiation.cc:240] Negotiation successful
0806 09:32:03.436894 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22550,"mutex_wait_us":2,"thread_start_us":23693,"threads_started":1}
I20260806 09:32:03.442879 9843 tablet_service.cc:1467] Processing CreateTablet for tablet bd5cd11c91154e56ad46d5768a63ba0e (DEFAULT_TABLE table=test-table [id=9796d0b25129453ab171c3cf21fc91d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:32:03.442891 10028 tablet_service.cc:1467] Processing CreateTablet for tablet bd5cd11c91154e56ad46d5768a63ba0e (DEFAULT_TABLE table=test-table [id=9796d0b25129453ab171c3cf21fc91d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:32:03.442891 9663 tablet_service.cc:1467] Processing CreateTablet for tablet bd5cd11c91154e56ad46d5768a63ba0e (DEFAULT_TABLE table=test-table [id=9796d0b25129453ab171c3cf21fc91d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:32:03.444193 9843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd5cd11c91154e56ad46d5768a63ba0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:32:03.444339 10028 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd5cd11c91154e56ad46d5768a63ba0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:32:03.444370 9663 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd5cd11c91154e56ad46d5768a63ba0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:32:03.456547 10155 tablet_bootstrap.cc:492] T bd5cd11c91154e56ad46d5768a63ba0e P 7702d30841eb48b79a91eaaeaf45e426: Bootstrap starting.
I20260806 09:32:03.460129 10155 tablet_bootstrap.cc:654] T bd5cd11c91154e56ad46d5768a63ba0e P 7702d30841eb48b79a91eaaeaf45e426: Neither blocks nor log segments found. Creating new log.
I20260806 09:32:03.461375 10155 log.cc:824] T bd5cd11c91154e56ad46d5768a63ba0e P 7702d30841eb48b79a91eaaeaf45e426: Log is configured to *not* fsync() on all Append() calls
I20260806 09:32:03.464284 10155 tablet_bootstrap.cc:492] T bd5cd11c91154e56ad46d5768a63ba0e P 7702d30841eb48b79a91eaaeaf45e426: No bootstrap required, opened a new log
I20260806 09:32:03.464627 10155 ts_tablet_manager.cc:1397] T bd5cd11c91154e56ad46d5768a63ba0e P 7702d30841eb48b79a91eaaeaf45e426: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.002s
I20260806 09:32:03.465096 10157 tablet_bootstrap.cc:492] T bd5cd11c91154e56ad46d5768a63ba0e P 339f163711b143c6a6a210b47915fb59: Bootstrap starting.
I20260806 09:32:03.465422 10156 tablet_bootstrap.cc:492] T bd5cd11c91154e56ad46d5768a63ba0e P 5425fb70daaf4f5e8bf84f5e5143e89f: Bootstrap starting.
I20260806 09:32:03.468822 10156 tablet_bootstrap.cc:654] T bd5cd11c91154e56ad46d5768a63ba0e P 5425fb70daaf4f5e8bf84f5e5143e89f: Neither blocks nor log segments found. Creating new log.
I20260806 09:32:03.468822 10157 tablet_bootstrap.cc:654] T bd5cd11c91154e56ad46d5768a63ba0e P 339f163711b143c6a6a210b47915fb59: Neither blocks nor log segments found. Creating new log.
I20260806 09:32:03.470098 10156 log.cc:824] T bd5cd11c91154e56ad46d5768a63ba0e P 5425fb70daaf4f5e8bf84f5e5143e89f: Log is configured to *not* fsync() on all Append() calls
I20260806 09:32:03.470103 10157 log.cc:824] T bd5cd11c91154e56ad46d5768a63ba0e P 339f163711b143c6a6a210b47915fb59: Log is configured to *not* fsync() on all Append() calls
I20260806 09:32:03.472877 10157 tablet_bootstrap.cc:492] T bd5cd11c91154e56ad46d5768a63ba0e P 339f163711b143c6a6a210b47915fb59: No bootstrap required, opened a new log
I20260806 09:32:03.473133 10157 ts_tablet_manager.cc:1397] T bd5cd11c91154e56ad46d5768a63ba0e P 339f163711b143c6a6a210b47915fb59: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260806 09:32:03.473312 10156 tablet_bootstrap.cc:492] T bd5cd11c91154e56ad46d5768a63ba0e P 5425fb70daaf4f5e8bf84f5e5143e89f: No bootstrap required, opened a new log
I20260806 09:32:03.473577 10156 ts_tablet_manager.cc:1397] T bd5cd11c91154e56ad46d5768a63ba0e P 5425fb70daaf4f5e8bf84f5e5143e89f: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.000s
I20260806 09:32:03.474073 10155 raft_consensus.cc:348] T bd5cd11c91154e56ad46d5768a63ba0e P 7702d30841eb48b79a91eaaeaf45e426 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7702d30841eb48b79a91eaaeaf45e426" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 37455 } } peers { permanent_uuid: "5425fb70daaf4f5e8bf84f5e5143e89f" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 46039 } } peers { permanent_uuid: "339f163711b143c6a6a210b47915fb59" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 39849 } }
I20260806 09:32:03.474538 10155 raft_consensus.cc:374] T bd5cd11c91154e56ad46d5768a63ba0e P 7702d30841eb48b79a91eaaeaf45e426 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:32:03.474649 10155 raft_consensus.cc:729] T bd5cd11c91154e56ad46d5768a63ba0e P 7702d30841eb48b79a91eaaeaf45e426 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7702d30841eb48b79a91eaaeaf45e426, State: Initialized, Role: FOLLOWER
I20260806 09:32:03.475397 10155 consensus_queue.cc:260] T bd5cd11c91154e56ad46d5768a63ba0e P 7702d30841eb48b79a91eaaeaf45e426 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7702d30841eb48b79a91eaaeaf45e426" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 37455 } } peers { permanent_uuid: "5425fb70daaf4f5e8bf84f5e5143e89f" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 46039 } } peers { permanent_uuid: "339f163711b143c6a6a210b47915fb59" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 39849 } }
I20260806 09:32:03.476663 9935 heartbeater.cc:503] Master 127.20.113.62:44265 was elected leader, sending a full tablet report...
I20260806 09:32:03.478601 10155 ts_tablet_manager.cc:1428] T bd5cd11c91154e56ad46d5768a63ba0e P 7702d30841eb48b79a91eaaeaf45e426: Time spent starting tablet: real 0.014s user 0.009s sys 0.002s
I20260806 09:32:03.482137 10157 raft_consensus.cc:348] T bd5cd11c91154e56ad46d5768a63ba0e P 339f163711b143c6a6a210b47915fb59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7702d30841eb48b79a91eaaeaf45e426" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 37455 } } peers { permanent_uuid: "5425fb70daaf4f5e8bf84f5e5143e89f" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 46039 } } peers { permanent_uuid: "339f163711b143c6a6a210b47915fb59" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 39849 } }
I20260806 09:32:03.482628 10157 raft_consensus.cc:374] T bd5cd11c91154e56ad46d5768a63ba0e P 339f163711b143c6a6a210b47915fb59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:32:03.482750 10157 raft_consensus.cc:729] T bd5cd11c91154e56ad46d5768a63ba0e P 339f163711b143c6a6a210b47915fb59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 339f163711b143c6a6a210b47915fb59, State: Initialized, Role: FOLLOWER
I20260806 09:32:03.482982 10156 raft_consensus.cc:348] T bd5cd11c91154e56ad46d5768a63ba0e P 5425fb70daaf4f5e8bf84f5e5143e89f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7702d30841eb48b79a91eaaeaf45e426" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 37455 } } peers { permanent_uuid: "5425fb70daaf4f5e8bf84f5e5143e89f" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 46039 } } peers { permanent_uuid: "339f163711b143c6a6a210b47915fb59" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 39849 } }
I20260806 09:32:03.483311 10156 raft_consensus.cc:374] T bd5cd11c91154e56ad46d5768a63ba0e P 5425fb70daaf4f5e8bf84f5e5143e89f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:32:03.483403 10156 raft_consensus.cc:729] T bd5cd11c91154e56ad46d5768a63ba0e P 5425fb70daaf4f5e8bf84f5e5143e89f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5425fb70daaf4f5e8bf84f5e5143e89f, State: Initialized, Role: FOLLOWER
I20260806 09:32:03.483422 10157 consensus_queue.cc:260] T bd5cd11c91154e56ad46d5768a63ba0e P 339f163711b143c6a6a210b47915fb59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7702d30841eb48b79a91eaaeaf45e426" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 37455 } } peers { permanent_uuid: "5425fb70daaf4f5e8bf84f5e5143e89f" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 46039 } } peers { permanent_uuid: "339f163711b143c6a6a210b47915fb59" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 39849 } }
I20260806 09:32:03.484012 10156 consensus_queue.cc:260] T bd5cd11c91154e56ad46d5768a63ba0e P 5425fb70daaf4f5e8bf84f5e5143e89f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7702d30841eb48b79a91eaaeaf45e426" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 37455 } } peers { permanent_uuid: "5425fb70daaf4f5e8bf84f5e5143e89f" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 46039 } } peers { permanent_uuid: "339f163711b143c6a6a210b47915fb59" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 39849 } }
I20260806 09:32:03.484642 10131 heartbeater.cc:503] Master 127.20.113.62:44265 was elected leader, sending a full tablet report...
I20260806 09:32:03.485306 10157 ts_tablet_manager.cc:1428] T bd5cd11c91154e56ad46d5768a63ba0e P 339f163711b143c6a6a210b47915fb59: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
I20260806 09:32:03.485620 10156 ts_tablet_manager.cc:1428] T bd5cd11c91154e56ad46d5768a63ba0e P 5425fb70daaf4f5e8bf84f5e5143e89f: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
W20260806 09:32:03.599084 10133 tablet.cc:2369] T bd5cd11c91154e56ad46d5768a63ba0e P 339f163711b143c6a6a210b47915fb59: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 09:32:03.612023 10164 raft_consensus.cc:482] T bd5cd11c91154e56ad46d5768a63ba0e P 7702d30841eb48b79a91eaaeaf45e426 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 09:32:03.612357 10164 raft_consensus.cc:504] T bd5cd11c91154e56ad46d5768a63ba0e P 7702d30841eb48b79a91eaaeaf45e426 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7702d30841eb48b79a91eaaeaf45e426" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 37455 } } peers { permanent_uuid: "5425fb70daaf4f5e8bf84f5e5143e89f" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 46039 } } peers { permanent_uuid: "339f163711b143c6a6a210b47915fb59" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 39849 } }
I20260806 09:32:03.614028 10164 leader_election.cc:288] T bd5cd11c91154e56ad46d5768a63ba0e P 7702d30841eb48b79a91eaaeaf45e426 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5425fb70daaf4f5e8bf84f5e5143e89f (127.20.113.1:46039), 339f163711b143c6a6a210b47915fb59 (127.20.113.3:39849)
I20260806 09:32:03.623066 10151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:03.614521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.2:43545 (local address 127.20.113.1:46039)
0806 09:32:03.614755 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0806 09:32:03.614763 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 09:32:03.615509 (+ 746us) server_negotiation.cc:407] Connection header received
0806 09:32:03.615753 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:32:03.615757 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:32:03.615819 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:32:03.615929 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:32:03.616795 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:03.617617 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:03.620624 (+ 3007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:03.621891 (+ 1267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:03.622054 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:32:03.622128 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 09:32:03.622663 (+ 535us) server_negotiation.cc:299] Negotiation successful
0806 09:32:03.622790 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260806 09:32:03.623052 10152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:03.614620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.2:56523 (local address 127.20.113.3:39849)
0806 09:32:03.614856 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0806 09:32:03.614866 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 09:32:03.615587 (+ 721us) server_negotiation.cc:407] Connection header received
0806 09:32:03.615809 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:32:03.615813 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:32:03.615871 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:32:03.615968 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:32:03.616715 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:03.617527 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:03.620600 (+ 3073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:03.621906 (+ 1306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:03.622054 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:32:03.622129 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 09:32:03.622663 (+ 534us) server_negotiation.cc:299] Negotiation successful
0806 09:32:03.622792 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260806 09:32:03.623267 10169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:03.614482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.1:46039 (local address 127.20.113.2:43545)
0806 09:32:03.615383 (+ 901us) negotiation.cc:107] Waiting for socket to connect
0806 09:32:03.615407 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 09:32:03.615576 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:32:03.615945 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:32:03.615950 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:32:03.615974 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:32:03.616643 (+ 669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:32:03.616655 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:32:03.617733 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:32:03.617739 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:32:03.620396 (+ 2657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:32:03.620412 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:32:03.622188 (+ 1776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:32:03.622193 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:32:03.622406 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:32:03.622522 (+ 116us) client_negotiation.cc:769] Sending connection context
0806 09:32:03.622618 (+ 96us) client_negotiation.cc:240] Negotiation successful
0806 09:32:03.622772 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":759,"mutex_wait_us":210,"thread_start_us":504,"threads_started":1}
I20260806 09:32:03.623322 10170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:03.614531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.3:39849 (local address 127.20.113.2:56523)
0806 09:32:03.615481 (+ 950us) negotiation.cc:107] Waiting for socket to connect
0806 09:32:03.615495 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 09:32:03.615646 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:32:03.615982 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:32:03.615986 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:32:03.616008 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:32:03.616513 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:32:03.616524 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:32:03.617656 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:32:03.617662 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:32:03.620396 (+ 2734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:32:03.620415 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:32:03.622179 (+ 1764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:32:03.622186 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:32:03.622402 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:32:03.622522 (+ 120us) client_negotiation.cc:769] Sending connection context
0806 09:32:03.622618 (+ 96us) client_negotiation.cc:240] Negotiation successful
0806 09:32:03.622772 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":801,"mutex_wait_us":195,"thread_start_us":574,"threads_started":1}
I20260806 09:32:03.624641 9690 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd5cd11c91154e56ad46d5768a63ba0e" candidate_uuid: "7702d30841eb48b79a91eaaeaf45e426" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5425fb70daaf4f5e8bf84f5e5143e89f" is_pre_election: true
I20260806 09:32:03.624651 10055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd5cd11c91154e56ad46d5768a63ba0e" candidate_uuid: "7702d30841eb48b79a91eaaeaf45e426" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "339f163711b143c6a6a210b47915fb59" is_pre_election: true
I20260806 09:32:03.625258 9690 raft_consensus.cc:2454] T bd5cd11c91154e56ad46d5768a63ba0e P 5425fb70daaf4f5e8bf84f5e5143e89f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7702d30841eb48b79a91eaaeaf45e426 in term 0.
I20260806 09:32:03.625258 10055 raft_consensus.cc:2454] T bd5cd11c91154e56ad46d5768a63ba0e P 339f163711b143c6a6a210b47915fb59 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7702d30841eb48b79a91eaaeaf45e426 in term 0.
I20260806 09:32:03.626044 9788 leader_election.cc:302] T bd5cd11c91154e56ad46d5768a63ba0e P 7702d30841eb48b79a91eaaeaf45e426 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 339f163711b143c6a6a210b47915fb59, 7702d30841eb48b79a91eaaeaf45e426; no voters:
I20260806 09:32:03.626428 10164 raft_consensus.cc:2781] T bd5cd11c91154e56ad46d5768a63ba0e P 7702d30841eb48b79a91eaaeaf45e426 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 09:32:03.626582 10164 raft_consensus.cc:482] T bd5cd11c91154e56ad46d5768a63ba0e P 7702d30841eb48b79a91eaaeaf45e426 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 09:32:03.626690 10164 raft_consensus.cc:3037] T bd5cd11c91154e56ad46d5768a63ba0e P 7702d30841eb48b79a91eaaeaf45e426 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:32:03.628957 10164 raft_consensus.cc:504] T bd5cd11c91154e56ad46d5768a63ba0e P 7702d30841eb48b79a91eaaeaf45e426 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7702d30841eb48b79a91eaaeaf45e426" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 37455 } } peers { permanent_uuid: "5425fb70daaf4f5e8bf84f5e5143e89f" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 46039 } } peers { permanent_uuid: "339f163711b143c6a6a210b47915fb59" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 39849 } }
I20260806 09:32:03.629537 10164 leader_election.cc:288] T bd5cd11c91154e56ad46d5768a63ba0e P 7702d30841eb48b79a91eaaeaf45e426 [CANDIDATE]: Term 1 election: Requested vote from peers 5425fb70daaf4f5e8bf84f5e5143e89f (127.20.113.1:46039), 339f163711b143c6a6a210b47915fb59 (127.20.113.3:39849)
I20260806 09:32:03.629909 9690 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd5cd11c91154e56ad46d5768a63ba0e" candidate_uuid: "7702d30841eb48b79a91eaaeaf45e426" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5425fb70daaf4f5e8bf84f5e5143e89f"
I20260806 09:32:03.630053 10055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd5cd11c91154e56ad46d5768a63ba0e" candidate_uuid: "7702d30841eb48b79a91eaaeaf45e426" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "339f163711b143c6a6a210b47915fb59"
I20260806 09:32:03.630107 9690 raft_consensus.cc:3037] T bd5cd11c91154e56ad46d5768a63ba0e P 5425fb70daaf4f5e8bf84f5e5143e89f [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:32:03.630260 10055 raft_consensus.cc:3037] T bd5cd11c91154e56ad46d5768a63ba0e P 339f163711b143c6a6a210b47915fb59 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:32:03.632220 9690 raft_consensus.cc:2454] T bd5cd11c91154e56ad46d5768a63ba0e P 5425fb70daaf4f5e8bf84f5e5143e89f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7702d30841eb48b79a91eaaeaf45e426 in term 1.
I20260806 09:32:03.632587 10055 raft_consensus.cc:2454] T bd5cd11c91154e56ad46d5768a63ba0e P 339f163711b143c6a6a210b47915fb59 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7702d30841eb48b79a91eaaeaf45e426 in term 1.
I20260806 09:32:03.632701 9787 leader_election.cc:302] T bd5cd11c91154e56ad46d5768a63ba0e P 7702d30841eb48b79a91eaaeaf45e426 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5425fb70daaf4f5e8bf84f5e5143e89f, 7702d30841eb48b79a91eaaeaf45e426; no voters:
I20260806 09:32:03.633183 10164 raft_consensus.cc:2781] T bd5cd11c91154e56ad46d5768a63ba0e P 7702d30841eb48b79a91eaaeaf45e426 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:32:03.633643 10164 raft_consensus.cc:686] T bd5cd11c91154e56ad46d5768a63ba0e P 7702d30841eb48b79a91eaaeaf45e426 [term 1 LEADER]: Becoming Leader. State: Replica: 7702d30841eb48b79a91eaaeaf45e426, State: Running, Role: LEADER
I20260806 09:32:03.634150 10164 consensus_queue.cc:237] T bd5cd11c91154e56ad46d5768a63ba0e P 7702d30841eb48b79a91eaaeaf45e426 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7702d30841eb48b79a91eaaeaf45e426" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 37455 } } peers { permanent_uuid: "5425fb70daaf4f5e8bf84f5e5143e89f" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 46039 } } peers { permanent_uuid: "339f163711b143c6a6a210b47915fb59" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 39849 } }
W20260806 09:32:03.637591 9936 tablet.cc:2369] T bd5cd11c91154e56ad46d5768a63ba0e P 7702d30841eb48b79a91eaaeaf45e426: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 09:32:03.639016 9469 catalog_manager.cc:5495] T bd5cd11c91154e56ad46d5768a63ba0e P 7702d30841eb48b79a91eaaeaf45e426 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7702d30841eb48b79a91eaaeaf45e426 (127.20.113.2). New cstate: current_term: 1 leader_uuid: "7702d30841eb48b79a91eaaeaf45e426" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7702d30841eb48b79a91eaaeaf45e426" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 37455 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5425fb70daaf4f5e8bf84f5e5143e89f" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 46039 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "339f163711b143c6a6a210b47915fb59" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 39849 } health_report { overall_health: UNKNOWN } } }
W20260806 09:32:03.647477 9755 tablet.cc:2369] T bd5cd11c91154e56ad46d5768a63ba0e P 5425fb70daaf4f5e8bf84f5e5143e89f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 09:32:03.683959 10146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:03.678613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58472 (local address 127.20.113.2:37455)
0806 09:32:03.678804 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0806 09:32:03.678808 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 09:32:03.678848 (+ 40us) server_negotiation.cc:407] Connection header received
0806 09:32:03.679056 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:32:03.679060 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:32:03.679151 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:32:03.679273 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 09:32:03.679618 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:03.680404 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:03.681847 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:03.682238 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:03.682325 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:32:03.682883 (+ 558us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 09:32:03.683390 (+ 507us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 09:32:03.683508 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0806 09:32:03.683593 (+ 85us) server_negotiation.cc:299] Negotiation successful
0806 09:32:03.683657 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 09:32:03.693249 10055 raft_consensus.cc:1261] T bd5cd11c91154e56ad46d5768a63ba0e P 339f163711b143c6a6a210b47915fb59 [term 1 FOLLOWER]: Refusing update from remote peer 7702d30841eb48b79a91eaaeaf45e426: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 09:32:03.693249 9690 raft_consensus.cc:1261] T bd5cd11c91154e56ad46d5768a63ba0e P 5425fb70daaf4f5e8bf84f5e5143e89f [term 1 FOLLOWER]: Refusing update from remote peer 7702d30841eb48b79a91eaaeaf45e426: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 09:32:03.694336 10171 consensus_queue.cc:1035] T bd5cd11c91154e56ad46d5768a63ba0e P 7702d30841eb48b79a91eaaeaf45e426 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5425fb70daaf4f5e8bf84f5e5143e89f" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 46039 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 09:32:03.694677 10164 consensus_queue.cc:1035] T bd5cd11c91154e56ad46d5768a63ba0e P 7702d30841eb48b79a91eaaeaf45e426 [LEADER]: Connected to new peer: Peer: permanent_uuid: "339f163711b143c6a6a210b47915fb59" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 39849 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 09:32:03.720670 9469 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:44718:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 09:32:03.720927 9469 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:44718:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 09:32:03.724457 9469 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2ebb3c6330eb44f9b537d7c3007f0271: Sending DeleteTablet for 3 replicas of tablet bd5cd11c91154e56ad46d5768a63ba0e
I20260806 09:32:03.725548 9843 tablet_service.cc:1514] Processing DeleteTablet for tablet bd5cd11c91154e56ad46d5768a63ba0e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:32:03 UTC) from {username='kudu', principal='kudu/127.20.113.62@KRBTEST.COM'} at 127.0.0.1:58458
I20260806 09:32:03.725690 9663 tablet_service.cc:1514] Processing DeleteTablet for tablet bd5cd11c91154e56ad46d5768a63ba0e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:32:03 UTC) from {username='kudu', principal='kudu/127.20.113.62@KRBTEST.COM'} at 127.0.0.1:44700
I20260806 09:32:03.725899 10028 tablet_service.cc:1514] Processing DeleteTablet for tablet bd5cd11c91154e56ad46d5768a63ba0e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:32:03 UTC) from {username='kudu', principal='kudu/127.20.113.62@KRBTEST.COM'} at 127.0.0.1:44670
I20260806 09:32:03.726964 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 9532
I20260806 09:32:03.742861 10192 tablet_replica.cc:330] T bd5cd11c91154e56ad46d5768a63ba0e P 5425fb70daaf4f5e8bf84f5e5143e89f: stopping tablet replica
I20260806 09:32:03.744333 10192 raft_consensus.cc:2229] T bd5cd11c91154e56ad46d5768a63ba0e P 5425fb70daaf4f5e8bf84f5e5143e89f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 09:32:03.745302 10194 tablet_replica.cc:330] T bd5cd11c91154e56ad46d5768a63ba0e P 339f163711b143c6a6a210b47915fb59: stopping tablet replica
I20260806 09:32:03.746138 10192 raft_consensus.cc:2258] T bd5cd11c91154e56ad46d5768a63ba0e P 5425fb70daaf4f5e8bf84f5e5143e89f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 09:32:03.746750 10194 raft_consensus.cc:2229] T bd5cd11c91154e56ad46d5768a63ba0e P 339f163711b143c6a6a210b47915fb59 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 09:32:03.747687 10194 raft_consensus.cc:2258] T bd5cd11c91154e56ad46d5768a63ba0e P 339f163711b143c6a6a210b47915fb59 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 09:32:03.748472 10151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:03.727840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44706 (local address 127.20.113.1:46039)
0806 09:32:03.728017 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0806 09:32:03.728023 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:32:03.728267 (+ 244us) server_negotiation.cc:407] Connection header received
0806 09:32:03.728505 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:32:03.728511 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:32:03.728594 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:32:03.728708 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:32:03.730996 (+ 2288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:03.732190 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:03.734703 (+ 2513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:03.735377 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:03.735520 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:32:03.739385 (+ 3865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:32:03.739405 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:32:03.739424 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:32:03.739470 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:32:03.743955 (+ 4485us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:32:03.744940 (+ 985us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:32:03.744952 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:32:03.744956 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:32:03.745132 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:32:03.745697 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:32:03.745702 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:32:03.745706 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:32:03.746222 (+ 516us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:32:03.746543 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0806 09:32:03.747764 (+ 1221us) server_negotiation.cc:299] Negotiation successful
0806 09:32:03.747991 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 09:32:03.749662 10191 tablet_replica.cc:330] T bd5cd11c91154e56ad46d5768a63ba0e P 7702d30841eb48b79a91eaaeaf45e426: stopping tablet replica
I20260806 09:32:03.749975 9745 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:32:03.750149 10191 raft_consensus.cc:2229] T bd5cd11c91154e56ad46d5768a63ba0e P 7702d30841eb48b79a91eaaeaf45e426 [term 1 LEADER]: Raft consensus shutting down.
I20260806 09:32:03.750910 10191 raft_consensus.cc:2258] T bd5cd11c91154e56ad46d5768a63ba0e P 7702d30841eb48b79a91eaaeaf45e426 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 09:32:03.752861 10191 ts_tablet_manager.cc:1905] T bd5cd11c91154e56ad46d5768a63ba0e P 7702d30841eb48b79a91eaaeaf45e426: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 09:32:03.753288 10194 ts_tablet_manager.cc:1905] T bd5cd11c91154e56ad46d5768a63ba0e P 339f163711b143c6a6a210b47915fb59: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 09:32:03.760768 10191 ts_tablet_manager.cc:1918] T bd5cd11c91154e56ad46d5768a63ba0e P 7702d30841eb48b79a91eaaeaf45e426: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 09:32:03.761049 10191 log.cc:1196] T bd5cd11c91154e56ad46d5768a63ba0e P 7702d30841eb48b79a91eaaeaf45e426: Deleting WAL directory at /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-1/wal/wals/bd5cd11c91154e56ad46d5768a63ba0e
I20260806 09:32:03.761763 10191 ts_tablet_manager.cc:1939] T bd5cd11c91154e56ad46d5768a63ba0e P 7702d30841eb48b79a91eaaeaf45e426: Deleting consensus metadata
I20260806 09:32:03.762655 10194 ts_tablet_manager.cc:1918] T bd5cd11c91154e56ad46d5768a63ba0e P 339f163711b143c6a6a210b47915fb59: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 09:32:03.762946 10194 log.cc:1196] T bd5cd11c91154e56ad46d5768a63ba0e P 339f163711b143c6a6a210b47915fb59: Deleting WAL directory at /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-2/wal/wals/bd5cd11c91154e56ad46d5768a63ba0e
I20260806 09:32:03.763701 10194 ts_tablet_manager.cc:1939] T bd5cd11c91154e56ad46d5768a63ba0e P 339f163711b143c6a6a210b47915fb59: Deleting consensus metadata
I20260806 09:32:03.763811 9449 catalog_manager.cc:4841] TS 7702d30841eb48b79a91eaaeaf45e426 (127.20.113.2:37455): tablet bd5cd11c91154e56ad46d5768a63ba0e (table test-table [id=9796d0b25129453ab171c3cf21fc91d6]) successfully deleted
I20260806 09:32:03.765812 9452 catalog_manager.cc:4841] TS 339f163711b143c6a6a210b47915fb59 (127.20.113.3:39849): tablet bd5cd11c91154e56ad46d5768a63ba0e (table test-table [id=9796d0b25129453ab171c3cf21fc91d6]) successfully deleted
I20260806 09:32:03.750363 10192 ts_tablet_manager.cc:1905] T bd5cd11c91154e56ad46d5768a63ba0e P 5425fb70daaf4f5e8bf84f5e5143e89f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 09:32:03.915863 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 9532
W20260806 09:32:03.944932 9451 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.20.113.1:46039 (error 108)
I20260806 09:32:03.945287 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 9761
I20260806 09:32:03.949489 10198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:03.946750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.1:46039 (local address 127.0.0.1:44712)
0806 09:32:03.949101 (+ 2351us) negotiation.cc:107] Waiting for socket to connect
0806 09:32:03.949240 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.1:46039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2215,"thread_start_us":2115,"threads_started":1}
W20260806 09:32:03.949779 9451 catalog_manager.cc:4584] TS 5425fb70daaf4f5e8bf84f5e5143e89f (127.20.113.1:46039): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet bd5cd11c91154e56ad46d5768a63ba0e: Network error: Client connection negotiation failed: client connection to 127.20.113.1:46039: connect: Connection refused (error 111)
I20260806 09:32:03.962409 10146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:03.946245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58476 (local address 127.20.113.2:37455)
0806 09:32:03.946469 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0806 09:32:03.946477 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 09:32:03.946502 (+ 25us) server_negotiation.cc:407] Connection header received
0806 09:32:03.946664 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:32:03.946670 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:32:03.946758 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:32:03.946935 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:32:03.948792 (+ 1857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:03.949882 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:03.951325 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:03.952045 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:03.952190 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:32:03.955130 (+ 2940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:32:03.955147 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:32:03.955165 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:32:03.955206 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:32:03.959393 (+ 4187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:32:03.960128 (+ 735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:32:03.960138 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:32:03.960142 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:32:03.960359 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:32:03.960750 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:32:03.960755 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:32:03.960758 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:32:03.961243 (+ 485us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:32:03.961596 (+ 353us) server_negotiation.cc:1035] Waiting for connection context
0806 09:32:03.961773 (+ 177us) server_negotiation.cc:299] Negotiation successful
0806 09:32:03.962002 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260806 09:32:03.963184 9928 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:32:04.003410 10198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:04.002854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.1:46039 (local address 127.0.0.1:44720)
0806 09:32:04.003098 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0806 09:32:04.003222 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.1:46039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260806 09:32:04.004598 10198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:04.004140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.1:46039 (local address 127.0.0.1:44732)
0806 09:32:04.004341 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0806 09:32:04.004427 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.1:46039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260806 09:32:04.065414 10198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:04.064836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.1:46039 (local address 127.0.0.1:44744)
0806 09:32:04.065071 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0806 09:32:04.065215 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.1:46039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260806 09:32:04.066586 10198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:04.066166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.1:46039 (local address 127.0.0.1:44746)
0806 09:32:04.066336 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0806 09:32:04.066421 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.1:46039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260806 09:32:04.082592 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 9761
I20260806 09:32:04.113483 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 9939
I20260806 09:32:04.127115 10152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:04.114226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44682 (local address 127.20.113.3:39849)
0806 09:32:04.114427 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0806 09:32:04.114432 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:32:04.114451 (+ 19us) server_negotiation.cc:407] Connection header received
0806 09:32:04.114648 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:32:04.114652 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:32:04.114733 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:32:04.114847 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:32:04.116287 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:04.117067 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:04.118362 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:04.118762 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:04.118864 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:32:04.121489 (+ 2625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:32:04.121503 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:32:04.121518 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:32:04.121551 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:32:04.124423 (+ 2872us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:32:04.125067 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:32:04.125074 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:32:04.125076 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:32:04.125217 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:32:04.125545 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:32:04.125548 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:32:04.125550 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:32:04.125885 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:32:04.126112 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0806 09:32:04.126521 (+ 409us) server_negotiation.cc:299] Negotiation successful
0806 09:32:04.126701 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260806 09:32:04.127863 10119 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:32:04.147539 10198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:04.147077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.1:46039 (local address 127.0.0.1:44762)
0806 09:32:04.147289 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0806 09:32:04.147380 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.1:46039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260806 09:32:04.148801 10198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:04.148370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.1:46039 (local address 127.0.0.1:44764)
0806 09:32:04.148584 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0806 09:32:04.148658 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.1:46039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123}
I20260806 09:32:04.219666 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 9939
I20260806 09:32:04.248509 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 9423
I20260806 09:32:04.249884 9522 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:32:04.366931 10198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:04.366423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.1:46039 (local address 127.0.0.1:44772)
0806 09:32:04.366631 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0806 09:32:04.366750 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.1:46039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260806 09:32:04.366958 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 9423
I20260806 09:32:04.398041 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.113.62:44265
--webserver_interface=127.20.113.62
--webserver_port=34199
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab
--principal=kudu/127.20.113.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:46541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.113.62:44265
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:32:04.744612 10203 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:32:04.744900 10203 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:32:04.744990 10203 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:32:04.751955 10203 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 09:32:04.752056 10203 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:32:04.752113 10203 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:32:04.752169 10203 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 09:32:04.752211 10203 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 09:32:04.761549 10203 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:46541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.113.62:44265
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab
--principal=kudu/127.20.113.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.113.62:44265
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.113.62
--webserver_port=34199
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:32:04.762773 10203 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:32:04.764030 10203 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:32:04.775944 10209 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:32:04.776018 10210 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:32:04.776058 10213 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:32:04.777123 10203 server_base.cc:1034] running on GCE node
I20260806 09:32:04.777839 10203 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:32:04.779141 10203 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:32:04.780391 10203 hybrid_clock.cc:648] HybridClock initialized: now 1786008724780319 us; error 57 us; skew 500 ppm
I20260806 09:32:04.780737 10203 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:32:04 dist-test-slave-45dx krb5kdc[8615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008724, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:32:04.783871 10203 init.cc:377] Logged in from keytab as kudu/127.20.113.62@KRBTEST.COM (short username kudu)
I20260806 09:32:04.785338 10203 webserver.cc:460] Webserver started at http://127.20.113.62:34199/ using document root <none> and password file <none>
I20260806 09:32:04.786031 10203 fs_manager.cc:359] Metadata directory not provided
I20260806 09:32:04.786168 10203 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:32:04.791476 10203 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.002s
I20260806 09:32:04.794342 10226 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:32:04.795647 10203 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260806 09:32:04.795861 10203 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/master-0/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/master-0/wal
uuid: "2ebb3c6330eb44f9b537d7c3007f0271"
format_stamp: "Formatted at 2026-08-06 09:31:58 on dist-test-slave-45dx"
server_key: "05ba32b4bb156010031014cc0f150aea"
server_key_iv: "9dbb2824c88d4631079ed6f5d6310224"
server_key_version: "encryptionkey@0"
I20260806 09:32:04.796913 10203 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:32:04.812453 10203 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:32:04.816139 10203 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:32:04.816462 10203 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:32:04.835028 10203 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.62:44265
I20260806 09:32:04.835052 10299 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.62:44265 every 8 connection(s)
I20260806 09:32:04.836577 10203 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
I20260806 09:32:04.838282 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 10203
I20260806 09:32:04.839088 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.113.1:46039
--local_ip_for_outbound_sockets=127.20.113.1
--tserver_master_addrs=127.20.113.62:44265
--webserver_port=45327
--webserver_interface=127.20.113.1
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.1.keytab
--principal=kudu/127.20.113.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:46541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 09:32:04.843551 10301 sys_catalog.cc:263] Verifying existing consensus state
I20260806 09:32:04.846728 10301 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ebb3c6330eb44f9b537d7c3007f0271: Bootstrap starting.
I20260806 09:32:04.867507 10301 log.cc:824] T 00000000000000000000000000000000 P 2ebb3c6330eb44f9b537d7c3007f0271: Log is configured to *not* fsync() on all Append() calls
I20260806 09:32:04.883744 10301 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ebb3c6330eb44f9b537d7c3007f0271: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 09:32:04.884366 10301 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ebb3c6330eb44f9b537d7c3007f0271: Bootstrap complete.
I20260806 09:32:04.893700 10301 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2ebb3c6330eb44f9b537d7c3007f0271 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ebb3c6330eb44f9b537d7c3007f0271" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 44265 } }
I20260806 09:32:04.894580 10301 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2ebb3c6330eb44f9b537d7c3007f0271 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ebb3c6330eb44f9b537d7c3007f0271, State: Initialized, Role: FOLLOWER
I20260806 09:32:04.895308 10301 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2ebb3c6330eb44f9b537d7c3007f0271 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ebb3c6330eb44f9b537d7c3007f0271" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 44265 } }
I20260806 09:32:04.895556 10301 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2ebb3c6330eb44f9b537d7c3007f0271 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 09:32:04.895679 10301 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2ebb3c6330eb44f9b537d7c3007f0271 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 09:32:04.895882 10301 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2ebb3c6330eb44f9b537d7c3007f0271 [term 2 FOLLOWER]: Advancing to term 3
I20260806 09:32:04.899695 10301 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2ebb3c6330eb44f9b537d7c3007f0271 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ebb3c6330eb44f9b537d7c3007f0271" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 44265 } }
I20260806 09:32:04.900396 10301 leader_election.cc:302] T 00000000000000000000000000000000 P 2ebb3c6330eb44f9b537d7c3007f0271 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2ebb3c6330eb44f9b537d7c3007f0271; no voters:
I20260806 09:32:04.901129 10301 leader_election.cc:288] T 00000000000000000000000000000000 P 2ebb3c6330eb44f9b537d7c3007f0271 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260806 09:32:04.901262 10307 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2ebb3c6330eb44f9b537d7c3007f0271 [term 3 FOLLOWER]: Leader election won for term 3
I20260806 09:32:04.902122 10307 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2ebb3c6330eb44f9b537d7c3007f0271 [term 3 LEADER]: Becoming Leader. State: Replica: 2ebb3c6330eb44f9b537d7c3007f0271, State: Running, Role: LEADER
I20260806 09:32:04.902887 10307 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2ebb3c6330eb44f9b537d7c3007f0271 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ebb3c6330eb44f9b537d7c3007f0271" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 44265 } }
I20260806 09:32:04.904863 10307 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ebb3c6330eb44f9b537d7c3007f0271 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2ebb3c6330eb44f9b537d7c3007f0271. Latest consensus state: current_term: 3 leader_uuid: "2ebb3c6330eb44f9b537d7c3007f0271" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ebb3c6330eb44f9b537d7c3007f0271" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 44265 } } }
I20260806 09:32:04.905313 10307 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ebb3c6330eb44f9b537d7c3007f0271 [sys.catalog]: This master's current role is: LEADER
I20260806 09:32:04.905642 10310 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ebb3c6330eb44f9b537d7c3007f0271 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "2ebb3c6330eb44f9b537d7c3007f0271" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ebb3c6330eb44f9b537d7c3007f0271" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 44265 } } }
I20260806 09:32:04.905990 10310 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ebb3c6330eb44f9b537d7c3007f0271 [sys.catalog]: This master's current role is: LEADER
I20260806 09:32:04.909699 10301 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2ebb3c6330eb44f9b537d7c3007f0271 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 09:32:04.913626 10312 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 09:32:04.922995 10312 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 09:32:04.924611 10312 catalog_manager.cc:1259] Loaded cluster ID: 77281e58ac894f2c99060d83083d2db4
I20260806 09:32:04.924850 10312 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260806 09:32:04.927237 10312 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fe71eee19ad google::LogMessage::Fail() at ??:0
@ 0x7fe71eee488a google::LogMessage::SendToLog() at ??:0
@ 0x7fe71eee14de google::LogMessage::Flush() at ??:0
@ 0x7fe71eee2a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fe734918512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7fe734915976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fe73499d610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7fe73499d439 std::_Function_handler<>::_M_invoke() at ??:0
I20260806 09:32:04.973397 10334 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fe721b2d345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fe721b3091c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7fe721b30799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fe721b0beaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7fe724de56db start_thread at ??:0
@ 0x7fe71dd7b61f clone at ??:0
W20260806 09:32:05.234035 10303 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:32:05.234387 10303 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:32:05.234560 10303 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:32:05.242129 10303 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:32:05.242286 10303 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:32:05.242480 10303 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.113.1
I20260806 09:32:05.252143 10303 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:46541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.1.keytab
--principal=kudu/127.20.113.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.113.1:46039
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.113.1
--webserver_port=45327
--tserver_master_addrs=127.20.113.62:44265
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.113.1
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:32:05.253504 10303 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:32:05.254944 10303 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:32:05.268181 10358 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:32:05.268221 10364 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:32:05.268455 10362 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:32:05.269193 10303 server_base.cc:1034] running on GCE node
I20260806 09:32:05.269843 10303 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:32:05.270825 10303 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:32:05.272104 10303 hybrid_clock.cc:648] HybridClock initialized: now 1786008725272057 us; error 57 us; skew 500 ppm
I20260806 09:32:05.272487 10303 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:32:05 dist-test-slave-45dx krb5kdc[8615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008725, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:32:05.275749 10303 init.cc:377] Logged in from keytab as kudu/127.20.113.1@KRBTEST.COM (short username kudu)
I20260806 09:32:05.277284 10303 webserver.cc:460] Webserver started at http://127.20.113.1:45327/ using document root <none> and password file <none>
I20260806 09:32:05.278016 10303 fs_manager.cc:359] Metadata directory not provided
I20260806 09:32:05.278173 10303 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:32:05.283821 10303 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 09:32:05.287017 10382 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:32:05.288475 10303 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 09:32:05.288651 10303 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-0/wal
uuid: "5425fb70daaf4f5e8bf84f5e5143e89f"
format_stamp: "Formatted at 2026-08-06 09:31:59 on dist-test-slave-45dx"
server_key: "29d545bab1cc17611d12fb3650b096a5"
server_key_iv: "96d0310e72ed5b248b2fbe011b5f80c5"
server_key_version: "encryptionkey@0"
I20260806 09:32:05.289671 10303 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:32:05.314582 10303 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:32:05.318413 10303 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:32:05.318714 10303 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:32:05.319707 10303 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:32:05.321545 10391 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260806 09:32:05.326953 10303 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 09:32:05.327138 10303 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.001s sys 0.000s
I20260806 09:32:05.327299 10303 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 09:32:05.330660 10303 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 09:32:05.330786 10303 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.001s sys 0.000s
I20260806 09:32:05.331112 10391 tablet_bootstrap.cc:492] T bd5cd11c91154e56ad46d5768a63ba0e P 5425fb70daaf4f5e8bf84f5e5143e89f: Bootstrap starting.
I20260806 09:32:05.339531 10391 log.cc:824] T bd5cd11c91154e56ad46d5768a63ba0e P 5425fb70daaf4f5e8bf84f5e5143e89f: Log is configured to *not* fsync() on all Append() calls
I20260806 09:32:05.354950 10391 tablet_bootstrap.cc:492] T bd5cd11c91154e56ad46d5768a63ba0e P 5425fb70daaf4f5e8bf84f5e5143e89f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 09:32:05.355532 10391 tablet_bootstrap.cc:492] T bd5cd11c91154e56ad46d5768a63ba0e P 5425fb70daaf4f5e8bf84f5e5143e89f: Bootstrap complete.
I20260806 09:32:05.355964 10391 ts_tablet_manager.cc:1397] T bd5cd11c91154e56ad46d5768a63ba0e P 5425fb70daaf4f5e8bf84f5e5143e89f: Time spent bootstrapping tablet: real 0.025s user 0.020s sys 0.000s
I20260806 09:32:05.366925 10391 raft_consensus.cc:348] T bd5cd11c91154e56ad46d5768a63ba0e P 5425fb70daaf4f5e8bf84f5e5143e89f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7702d30841eb48b79a91eaaeaf45e426" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 37455 } } peers { permanent_uuid: "5425fb70daaf4f5e8bf84f5e5143e89f" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 46039 } } peers { permanent_uuid: "339f163711b143c6a6a210b47915fb59" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 39849 } }
I20260806 09:32:05.368259 10391 raft_consensus.cc:729] T bd5cd11c91154e56ad46d5768a63ba0e P 5425fb70daaf4f5e8bf84f5e5143e89f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5425fb70daaf4f5e8bf84f5e5143e89f, State: Initialized, Role: FOLLOWER
I20260806 09:32:05.369175 10391 consensus_queue.cc:260] T bd5cd11c91154e56ad46d5768a63ba0e P 5425fb70daaf4f5e8bf84f5e5143e89f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7702d30841eb48b79a91eaaeaf45e426" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 37455 } } peers { permanent_uuid: "5425fb70daaf4f5e8bf84f5e5143e89f" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 46039 } } peers { permanent_uuid: "339f163711b143c6a6a210b47915fb59" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 39849 } }
I20260806 09:32:05.370796 10391 ts_tablet_manager.cc:1428] T bd5cd11c91154e56ad46d5768a63ba0e P 5425fb70daaf4f5e8bf84f5e5143e89f: Time spent starting tablet: real 0.015s user 0.013s sys 0.004s
I20260806 09:32:05.380132 10303 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.1:46039
I20260806 09:32:05.380250 10540 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.1:46039 every 8 connection(s)
I20260806 09:32:05.381462 10303 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-0/data/info.pb
I20260806 09:32:05.385421 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 10303
I20260806 09:32:05.386253 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.113.2:37455
--local_ip_for_outbound_sockets=127.20.113.2
--tserver_master_addrs=127.20.113.62:44265
--webserver_port=45355
--webserver_interface=127.20.113.2
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.2.keytab
--principal=kudu/127.20.113.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:46541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 09:32:05.386809 10546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:05.384196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44265 (local address 127.20.113.1:42239)
0806 09:32:05.385324 (+ 1128us) negotiation.cc:107] Waiting for socket to connect
0806 09:32:05.385630 (+ 306us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:44265: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":800,"thread_start_us":494,"threads_started":1}
W20260806 09:32:05.387521 10542 heartbeater.cc:644] Failed to heartbeat to 127.20.113.62:44265 (0 consecutive failures): Network error: Failed to ping master at 127.20.113.62:44265: Client connection negotiation failed: client connection to 127.20.113.62:44265: connect: Connection refused (error 111)
I20260806 09:32:05.388473 10546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:05.388022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44265 (local address 127.20.113.1:40987)
0806 09:32:05.388203 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0806 09:32:05.388323 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:44265: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260806 09:32:05.389474 10546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:05.389014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44265 (local address 127.20.113.1:58225)
0806 09:32:05.389186 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0806 09:32:05.389310 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:44265: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260806 09:32:05.389734 10542 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 09:32:05.734771 10547 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:32:05.735052 10547 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:32:05.735152 10547 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:32:05.742617 10547 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:32:05.742751 10547 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:32:05.742964 10547 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.113.2
I20260806 09:32:05.752522 10547 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:46541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.2.keytab
--principal=kudu/127.20.113.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.113.2:37455
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.113.2
--webserver_port=45355
--tserver_master_addrs=127.20.113.62:44265
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.113.2
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:32:05.753844 10547 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:32:05.755230 10547 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:32:05.769033 10554 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:32:05.769166 10557 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:32:05.769174 10555 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:32:05.769876 10547 server_base.cc:1034] running on GCE node
I20260806 09:32:05.770490 10547 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:32:05.771427 10547 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:32:05.772655 10547 hybrid_clock.cc:648] HybridClock initialized: now 1786008725772608 us; error 39 us; skew 500 ppm
I20260806 09:32:05.772989 10547 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:32:05 dist-test-slave-45dx krb5kdc[8615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008725, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:32:05.776197 10547 init.cc:377] Logged in from keytab as kudu/127.20.113.2@KRBTEST.COM (short username kudu)
I20260806 09:32:05.777709 10547 webserver.cc:460] Webserver started at http://127.20.113.2:45355/ using document root <none> and password file <none>
I20260806 09:32:05.778553 10547 fs_manager.cc:359] Metadata directory not provided
I20260806 09:32:05.778695 10547 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:32:05.784192 10547 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260806 09:32:05.786969 10566 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:32:05.788400 10547 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 09:32:05.788597 10547 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-1/wal
uuid: "7702d30841eb48b79a91eaaeaf45e426"
format_stamp: "Formatted at 2026-08-06 09:31:59 on dist-test-slave-45dx"
server_key: "b4b9ffe9ba6a109c233ab2bf8d6dc65e"
server_key_iv: "31395436d915c43eeceff885ef75b388"
server_key_version: "encryptionkey@0"
I20260806 09:32:05.789728 10547 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:32:05.815626 10547 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:32:05.819784 10547 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:32:05.820094 10547 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:32:05.821208 10547 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:32:05.822944 10547 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:32:05.823071 10547 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:32:05.823187 10547 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:32:05.823271 10547 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:32:05.857650 10547 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.2:37455
I20260806 09:32:05.857741 10720 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.2:37455 every 8 connection(s)
I20260806 09:32:05.859144 10547 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-1/data/info.pb
I20260806 09:32:05.861346 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 10547
I20260806 09:32:05.861989 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.113.3:39849
--local_ip_for_outbound_sockets=127.20.113.3
--tserver_master_addrs=127.20.113.62:44265
--webserver_port=34351
--webserver_interface=127.20.113.3
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.3.keytab
--principal=kudu/127.20.113.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:46541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 09:32:05.864440 10729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:05.862348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44265 (local address 127.20.113.2:54849)
0806 09:32:05.863227 (+ 879us) negotiation.cc:107] Waiting for socket to connect
0806 09:32:05.863537 (+ 310us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:44265: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":589,"thread_start_us":346,"threads_started":1}
W20260806 09:32:05.865137 10723 heartbeater.cc:644] Failed to heartbeat to 127.20.113.62:44265 (0 consecutive failures): Network error: Failed to ping master at 127.20.113.62:44265: Client connection negotiation failed: client connection to 127.20.113.62:44265: connect: Connection refused (error 111)
I20260806 09:32:05.865999 10729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:05.865668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44265 (local address 127.20.113.2:54535)
0806 09:32:05.865831 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0806 09:32:05.865898 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:44265: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260806 09:32:05.866889 10729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:05.866562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44265 (local address 127.20.113.2:59827)
0806 09:32:05.866695 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0806 09:32:05.866768 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:44265: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260806 09:32:05.867143 10723 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 09:32:06.204294 10730 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:32:06.204582 10730 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:32:06.204675 10730 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:32:06.211606 10730 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:32:06.211701 10730 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:32:06.211808 10730 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.113.3
I20260806 09:32:06.221263 10730 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:46541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.3.keytab
--principal=kudu/127.20.113.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.113.3:39849
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.113.3
--webserver_port=34351
--tserver_master_addrs=127.20.113.62:44265
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.113.3
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:32:06.222481 10730 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:32:06.223780 10730 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:32:06.237361 10737 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:32:06.237318 10746 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:32:06.237396 10738 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:32:06.238186 10730 server_base.cc:1034] running on GCE node
I20260806 09:32:06.238867 10730 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:32:06.239774 10730 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:32:06.241042 10730 hybrid_clock.cc:648] HybridClock initialized: now 1786008726240954 us; error 69 us; skew 500 ppm
I20260806 09:32:06.241430 10730 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:32:06 dist-test-slave-45dx krb5kdc[8615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008726, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:32:06.244717 10730 init.cc:377] Logged in from keytab as kudu/127.20.113.3@KRBTEST.COM (short username kudu)
I20260806 09:32:06.246086 10730 webserver.cc:460] Webserver started at http://127.20.113.3:34351/ using document root <none> and password file <none>
I20260806 09:32:06.246748 10730 fs_manager.cc:359] Metadata directory not provided
I20260806 09:32:06.246906 10730 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:32:06.252048 10730 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260806 09:32:06.254865 10758 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:32:06.256139 10730 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 09:32:06.256377 10730 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-2/wal
uuid: "339f163711b143c6a6a210b47915fb59"
format_stamp: "Formatted at 2026-08-06 09:32:00 on dist-test-slave-45dx"
server_key: "e8f9c570bff6a1289f44f97f65712e3a"
server_key_iv: "706b241d6e9475c962eac2b2696c4650"
server_key_version: "encryptionkey@0"
I20260806 09:32:06.257366 10730 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:32:06.282460 10730 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:32:06.286895 10730 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:32:06.287163 10730 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:32:06.288148 10730 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:32:06.289542 10730 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:32:06.289633 10730 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:32:06.289793 10730 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:32:06.289906 10730 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:32:06.323484 10730 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.3:39849
I20260806 09:32:06.323576 10917 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.3:39849 every 8 connection(s)
I20260806 09:32:06.325021 10730 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786008495763054-20932-0/minicluster-data/ts-2/data/info.pb
I20260806 09:32:06.325448 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 10730
I20260806 09:32:06.330348 10924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:06.328512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44265 (local address 127.20.113.3:50193)
0806 09:32:06.329238 (+ 726us) negotiation.cc:107] Waiting for socket to connect
0806 09:32:06.329484 (+ 246us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:44265: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":295,"threads_started":1}
W20260806 09:32:06.331043 10918 heartbeater.cc:644] Failed to heartbeat to 127.20.113.62:44265 (0 consecutive failures): Network error: Failed to ping master at 127.20.113.62:44265: Client connection negotiation failed: client connection to 127.20.113.62:44265: connect: Connection refused (error 111)
I20260806 09:32:06.331885 10924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:06.331557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44265 (local address 127.20.113.3:58251)
0806 09:32:06.331714 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0806 09:32:06.331786 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:44265: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260806 09:32:06.332828 10924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:06.332468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44265 (local address 127.20.113.3:60357)
0806 09:32:06.332605 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0806 09:32:06.332712 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:44265: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260806 09:32:06.333062 10918 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 09:32:06.391113 10925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:06.390272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44265 (local address 127.20.113.1:59131)
0806 09:32:06.390744 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0806 09:32:06.390894 (+ 150us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:44265: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":209,"threads_started":1}
I20260806 09:32:06.736351 10926 raft_consensus.cc:482] T bd5cd11c91154e56ad46d5768a63ba0e P 5425fb70daaf4f5e8bf84f5e5143e89f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 09:32:06.736636 10926 raft_consensus.cc:504] T bd5cd11c91154e56ad46d5768a63ba0e P 5425fb70daaf4f5e8bf84f5e5143e89f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7702d30841eb48b79a91eaaeaf45e426" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 37455 } } peers { permanent_uuid: "5425fb70daaf4f5e8bf84f5e5143e89f" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 46039 } } peers { permanent_uuid: "339f163711b143c6a6a210b47915fb59" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 39849 } }
I20260806 09:32:06.738233 10926 leader_election.cc:288] T bd5cd11c91154e56ad46d5768a63ba0e P 5425fb70daaf4f5e8bf84f5e5143e89f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7702d30841eb48b79a91eaaeaf45e426 (127.20.113.2:37455), 339f163711b143c6a6a210b47915fb59 (127.20.113.3:39849)
Aug 06 09:32:06 dist-test-slave-45dx krb5kdc[8615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008725, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.1@KRBTEST.COM for kudu/127.20.113.2@KRBTEST.COM
Aug 06 09:32:06 dist-test-slave-45dx krb5kdc[8615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008725, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.1@KRBTEST.COM for kudu/127.20.113.3@KRBTEST.COM
I20260806 09:32:06.764227 10933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:06.738932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.1:43137 (local address 127.20.113.3:39849)
0806 09:32:06.740792 (+ 1860us) server_negotiation.cc:206] Beginning negotiation
0806 09:32:06.740812 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 09:32:06.740869 (+ 57us) server_negotiation.cc:407] Connection header received
0806 09:32:06.743264 (+ 2395us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:32:06.743278 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:32:06.743766 (+ 488us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:32:06.744185 (+ 419us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:32:06.746666 (+ 2481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:06.748508 (+ 1842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:06.751193 (+ 2685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:06.752202 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:06.752445 (+ 243us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:32:06.757345 (+ 4900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:32:06.757352 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:32:06.757363 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:32:06.757396 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:32:06.760399 (+ 3003us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:32:06.761273 (+ 874us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:32:06.761286 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:32:06.761292 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:32:06.761444 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:32:06.761946 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:32:06.761952 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:32:06.761955 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:32:06.762470 (+ 515us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:32:06.762661 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0806 09:32:06.763404 (+ 743us) server_negotiation.cc:299] Negotiation successful
0806 09:32:06.763786 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1613,"thread_start_us":1343,"threads_started":1}
I20260806 09:32:06.764242 10932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:06.738936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.1:56747 (local address 127.20.113.2:37455)
0806 09:32:06.740951 (+ 2015us) server_negotiation.cc:206] Beginning negotiation
0806 09:32:06.740969 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 09:32:06.741823 (+ 854us) server_negotiation.cc:407] Connection header received
0806 09:32:06.744019 (+ 2196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:32:06.744032 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:32:06.744468 (+ 436us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:32:06.744797 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:32:06.746666 (+ 1869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:06.748508 (+ 1842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:06.751193 (+ 2685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:06.752202 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:06.752445 (+ 243us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:32:06.757113 (+ 4668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:32:06.757120 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:32:06.757132 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:32:06.757174 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:32:06.760273 (+ 3099us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:32:06.761298 (+ 1025us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:32:06.761315 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:32:06.761321 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:32:06.761472 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:32:06.761984 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:32:06.761990 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:32:06.761993 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:32:06.762547 (+ 554us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:32:06.762741 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0806 09:32:06.763406 (+ 665us) server_negotiation.cc:299] Negotiation successful
0806 09:32:06.763803 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1844,"thread_start_us":1741,"threads_started":1}
I20260806 09:32:06.764258 10925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:06.738588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.3:39849 (local address 127.20.113.1:43137)
0806 09:32:06.738928 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0806 09:32:06.738986 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0806 09:32:06.739682 (+ 696us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:32:06.744279 (+ 4597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:32:06.744286 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:32:06.745145 (+ 859us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:32:06.746322 (+ 1177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:32:06.746361 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:32:06.748742 (+ 2381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:32:06.748747 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:32:06.750941 (+ 2194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:32:06.750957 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:32:06.752502 (+ 1545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:32:06.752507 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:32:06.752879 (+ 372us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:32:06.753761 (+ 882us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:32:06.753802 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:32:06.757088 (+ 3286us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:32:06.760532 (+ 3444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:32:06.760537 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:32:06.760541 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:32:06.761136 (+ 595us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:32:06.761562 (+ 426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:32:06.761567 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:32:06.761570 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:32:06.761827 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:32:06.762617 (+ 790us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:32:06.762624 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:32:06.762922 (+ 298us) client_negotiation.cc:769] Sending connection context
0806 09:32:06.763190 (+ 268us) client_negotiation.cc:240] Negotiation successful
0806 09:32:06.763751 (+ 561us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209,"mutex_wait_us":303,"spinlock_wait_cycles":30976}
I20260806 09:32:06.764369 10928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:06.738608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.2:37455 (local address 127.20.113.1:56747)
0806 09:32:06.741668 (+ 3060us) negotiation.cc:107] Waiting for socket to connect
0806 09:32:06.741690 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 09:32:06.741856 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:32:06.744776 (+ 2920us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:32:06.744781 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:32:06.745450 (+ 669us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:32:06.746322 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:32:06.746361 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:32:06.748748 (+ 2387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:32:06.748752 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:32:06.750941 (+ 2189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:32:06.750957 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:32:06.752480 (+ 1523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:32:06.752489 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:32:06.752876 (+ 387us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:32:06.753772 (+ 896us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:32:06.753817 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:32:06.756815 (+ 2998us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:32:06.760450 (+ 3635us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:32:06.760459 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:32:06.760472 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:32:06.761161 (+ 689us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:32:06.761598 (+ 437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:32:06.761602 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:32:06.761606 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:32:06.761858 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:32:06.762699 (+ 841us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:32:06.762703 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:32:06.762916 (+ 213us) client_negotiation.cc:769] Sending connection context
0806 09:32:06.763192 (+ 276us) client_negotiation.cc:240] Negotiation successful
0806 09:32:06.763934 (+ 742us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2879,"mutex_wait_us":712,"spinlock_wait_cycles":6272,"thread_start_us":2462,"threads_started":1}
W20260806 09:32:06.767685 10383 leader_election.cc:341] T bd5cd11c91154e56ad46d5768a63ba0e P 5425fb70daaf4f5e8bf84f5e5143e89f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7702d30841eb48b79a91eaaeaf45e426 (127.20.113.2:37455): Not found: Tablet not found: bd5cd11c91154e56ad46d5768a63ba0e
W20260806 09:32:06.767927 10386 leader_election.cc:341] T bd5cd11c91154e56ad46d5768a63ba0e P 5425fb70daaf4f5e8bf84f5e5143e89f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 339f163711b143c6a6a210b47915fb59 (127.20.113.3:39849): Not found: Tablet not found: bd5cd11c91154e56ad46d5768a63ba0e
I20260806 09:32:06.768115 10386 leader_election.cc:302] T bd5cd11c91154e56ad46d5768a63ba0e P 5425fb70daaf4f5e8bf84f5e5143e89f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5425fb70daaf4f5e8bf84f5e5143e89f; no voters: 339f163711b143c6a6a210b47915fb59, 7702d30841eb48b79a91eaaeaf45e426
I20260806 09:32:06.768572 10926 raft_consensus.cc:2726] T bd5cd11c91154e56ad46d5768a63ba0e P 5425fb70daaf4f5e8bf84f5e5143e89f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 09:32:06.868721 10934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:06.867803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44265 (local address 127.20.113.2:50843)
0806 09:32:06.868249 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0806 09:32:06.868481 (+ 232us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:44265: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":201,"threads_started":1}
I20260806 09:32:07.334376 10935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:07.333556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44265 (local address 127.20.113.3:43161)
0806 09:32:07.334035 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0806 09:32:07.334167 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:44265: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":270,"threads_started":1}
I20260806 09:32:07.392849 10939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:07.392039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44265 (local address 127.20.113.1:51755)
0806 09:32:07.392576 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0806 09:32:07.392648 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:44265: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":326,"threads_started":1}
I20260806 09:32:07.870182 10941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:07.869533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44265 (local address 127.20.113.2:50833)
0806 09:32:07.869942 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0806 09:32:07.870034 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:44265: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":216,"threads_started":1}
I20260806 09:32:08.336159 10942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:08.335368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44265 (local address 127.20.113.3:50803)
0806 09:32:08.335863 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0806 09:32:08.335967 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:44265: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":266,"threads_started":1}
I20260806 09:32:08.394656 10943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:08.393789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44265 (local address 127.20.113.1:45339)
0806 09:32:08.394400 (+ 611us) negotiation.cc:107] Waiting for socket to connect
0806 09:32:08.394481 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:44265: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":353,"threads_started":1}
I20260806 09:32:08.680382 10944 raft_consensus.cc:482] T bd5cd11c91154e56ad46d5768a63ba0e P 5425fb70daaf4f5e8bf84f5e5143e89f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 09:32:08.680594 10944 raft_consensus.cc:504] T bd5cd11c91154e56ad46d5768a63ba0e P 5425fb70daaf4f5e8bf84f5e5143e89f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7702d30841eb48b79a91eaaeaf45e426" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 37455 } } peers { permanent_uuid: "5425fb70daaf4f5e8bf84f5e5143e89f" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 46039 } } peers { permanent_uuid: "339f163711b143c6a6a210b47915fb59" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 39849 } }
I20260806 09:32:08.681272 10944 leader_election.cc:288] T bd5cd11c91154e56ad46d5768a63ba0e P 5425fb70daaf4f5e8bf84f5e5143e89f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7702d30841eb48b79a91eaaeaf45e426 (127.20.113.2:37455), 339f163711b143c6a6a210b47915fb59 (127.20.113.3:39849)
W20260806 09:32:08.682274 10383 leader_election.cc:341] T bd5cd11c91154e56ad46d5768a63ba0e P 5425fb70daaf4f5e8bf84f5e5143e89f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7702d30841eb48b79a91eaaeaf45e426 (127.20.113.2:37455): Not found: Tablet not found: bd5cd11c91154e56ad46d5768a63ba0e
W20260806 09:32:08.682451 10386 leader_election.cc:341] T bd5cd11c91154e56ad46d5768a63ba0e P 5425fb70daaf4f5e8bf84f5e5143e89f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 339f163711b143c6a6a210b47915fb59 (127.20.113.3:39849): Not found: Tablet not found: bd5cd11c91154e56ad46d5768a63ba0e
I20260806 09:32:08.682559 10386 leader_election.cc:302] T bd5cd11c91154e56ad46d5768a63ba0e P 5425fb70daaf4f5e8bf84f5e5143e89f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5425fb70daaf4f5e8bf84f5e5143e89f; no voters: 339f163711b143c6a6a210b47915fb59, 7702d30841eb48b79a91eaaeaf45e426
I20260806 09:32:08.682830 10944 raft_consensus.cc:2726] T bd5cd11c91154e56ad46d5768a63ba0e P 5425fb70daaf4f5e8bf84f5e5143e89f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 09:32:08.871670 10946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:08.870987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44265 (local address 127.20.113.2:49165)
0806 09:32:08.871429 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0806 09:32:08.871511 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:44265: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":221,"threads_started":1}
I20260806 09:32:09.338008 10947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:09.337187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44265 (local address 127.20.113.3:38505)
0806 09:32:09.337719 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0806 09:32:09.337833 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:44265: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":293,"threads_started":1}
I20260806 09:32:09.396209 10949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:09.395544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44265 (local address 127.20.113.1:33861)
0806 09:32:09.395950 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0806 09:32:09.396046 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:44265: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":212,"threads_started":1}
I20260806 09:32:09.873390 10950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:09.872578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44265 (local address 127.20.113.2:49467)
0806 09:32:09.873119 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0806 09:32:09.873198 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:44265: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":308,"threads_started":1}
I20260806 09:32:10.339955 10951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:10.339150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44265 (local address 127.20.113.3:55661)
0806 09:32:10.339661 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0806 09:32:10.339758 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:44265: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":261,"threads_started":1}
I20260806 09:32:10.398066 10953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:10.397250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44265 (local address 127.20.113.1:56699)
0806 09:32:10.397766 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0806 09:32:10.397863 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:44265: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":261,"threads_started":1}
I20260806 09:32:10.875006 10955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:10.874338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44265 (local address 127.20.113.2:60839)
0806 09:32:10.874764 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0806 09:32:10.874844 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:44265: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":218,"threads_started":1}
I20260806 09:32:11.341684 10956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:11.340949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44265 (local address 127.20.113.3:52687)
0806 09:32:11.341387 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0806 09:32:11.341477 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:44265: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":241,"threads_started":1}
I20260806 09:32:11.399644 10957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:11.398916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44265 (local address 127.20.113.1:34217)
0806 09:32:11.399379 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0806 09:32:11.399467 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:44265: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":244,"threads_started":1}
I20260806 09:32:11.876731 10958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:11.875940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44265 (local address 127.20.113.2:56461)
0806 09:32:11.876474 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0806 09:32:11.876557 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:44265: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":439,"thread_start_us":268,"threads_started":1}
I20260806 09:32:12.010514 10959 raft_consensus.cc:482] T bd5cd11c91154e56ad46d5768a63ba0e P 5425fb70daaf4f5e8bf84f5e5143e89f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 09:32:12.010737 10959 raft_consensus.cc:504] T bd5cd11c91154e56ad46d5768a63ba0e P 5425fb70daaf4f5e8bf84f5e5143e89f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7702d30841eb48b79a91eaaeaf45e426" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 37455 } } peers { permanent_uuid: "5425fb70daaf4f5e8bf84f5e5143e89f" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 46039 } } peers { permanent_uuid: "339f163711b143c6a6a210b47915fb59" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 39849 } }
I20260806 09:32:12.011727 10959 leader_election.cc:288] T bd5cd11c91154e56ad46d5768a63ba0e P 5425fb70daaf4f5e8bf84f5e5143e89f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7702d30841eb48b79a91eaaeaf45e426 (127.20.113.2:37455), 339f163711b143c6a6a210b47915fb59 (127.20.113.3:39849)
W20260806 09:32:12.012460 10383 leader_election.cc:341] T bd5cd11c91154e56ad46d5768a63ba0e P 5425fb70daaf4f5e8bf84f5e5143e89f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7702d30841eb48b79a91eaaeaf45e426 (127.20.113.2:37455): Not found: Tablet not found: bd5cd11c91154e56ad46d5768a63ba0e
W20260806 09:32:12.012821 10386 leader_election.cc:341] T bd5cd11c91154e56ad46d5768a63ba0e P 5425fb70daaf4f5e8bf84f5e5143e89f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 339f163711b143c6a6a210b47915fb59 (127.20.113.3:39849): Not found: Tablet not found: bd5cd11c91154e56ad46d5768a63ba0e
I20260806 09:32:12.012959 10386 leader_election.cc:302] T bd5cd11c91154e56ad46d5768a63ba0e P 5425fb70daaf4f5e8bf84f5e5143e89f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5425fb70daaf4f5e8bf84f5e5143e89f; no voters: 339f163711b143c6a6a210b47915fb59, 7702d30841eb48b79a91eaaeaf45e426
I20260806 09:32:12.013254 10959 raft_consensus.cc:2726] T bd5cd11c91154e56ad46d5768a63ba0e P 5425fb70daaf4f5e8bf84f5e5143e89f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 09:32:12.343575 10961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:12.342715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44265 (local address 127.20.113.3:54651)
0806 09:32:12.343299 (+ 584us) negotiation.cc:107] Waiting for socket to connect
0806 09:32:12.343381 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:44265: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":372,"threads_started":1}
I20260806 09:32:12.401584 10962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:12.400787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44265 (local address 127.20.113.1:58103)
0806 09:32:12.401273 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0806 09:32:12.401377 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:44265: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":252,"threads_started":1}
I20260806 09:32:12.878506 10966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:12.877638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44265 (local address 127.20.113.2:37113)
0806 09:32:12.878179 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0806 09:32:12.878289 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:44265: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":239,"threads_started":1}
I20260806 09:32:13.345440 10967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:13.344574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44265 (local address 127.20.113.3:37263)
0806 09:32:13.345141 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0806 09:32:13.345226 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:44265: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":279,"threads_started":1}
I20260806 09:32:13.403345 10970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:13.402588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44265 (local address 127.20.113.1:58713)
0806 09:32:13.403089 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0806 09:32:13.403175 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:44265: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":269,"threads_started":1}
I20260806 09:32:13.880126 10971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:13.879436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44265 (local address 127.20.113.2:48211)
0806 09:32:13.879844 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0806 09:32:13.879931 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:44265: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":209,"threads_started":1}
I20260806 09:32:14.201002 10972 raft_consensus.cc:482] T bd5cd11c91154e56ad46d5768a63ba0e P 5425fb70daaf4f5e8bf84f5e5143e89f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 09:32:14.201244 10972 raft_consensus.cc:504] T bd5cd11c91154e56ad46d5768a63ba0e P 5425fb70daaf4f5e8bf84f5e5143e89f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7702d30841eb48b79a91eaaeaf45e426" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 37455 } } peers { permanent_uuid: "5425fb70daaf4f5e8bf84f5e5143e89f" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 46039 } } peers { permanent_uuid: "339f163711b143c6a6a210b47915fb59" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 39849 } }
I20260806 09:32:14.201998 10972 leader_election.cc:288] T bd5cd11c91154e56ad46d5768a63ba0e P 5425fb70daaf4f5e8bf84f5e5143e89f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7702d30841eb48b79a91eaaeaf45e426 (127.20.113.2:37455), 339f163711b143c6a6a210b47915fb59 (127.20.113.3:39849)
W20260806 09:32:14.202956 10383 leader_election.cc:341] T bd5cd11c91154e56ad46d5768a63ba0e P 5425fb70daaf4f5e8bf84f5e5143e89f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7702d30841eb48b79a91eaaeaf45e426 (127.20.113.2:37455): Not found: Tablet not found: bd5cd11c91154e56ad46d5768a63ba0e
W20260806 09:32:14.203159 10386 leader_election.cc:341] T bd5cd11c91154e56ad46d5768a63ba0e P 5425fb70daaf4f5e8bf84f5e5143e89f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 339f163711b143c6a6a210b47915fb59 (127.20.113.3:39849): Not found: Tablet not found: bd5cd11c91154e56ad46d5768a63ba0e
I20260806 09:32:14.203299 10386 leader_election.cc:302] T bd5cd11c91154e56ad46d5768a63ba0e P 5425fb70daaf4f5e8bf84f5e5143e89f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5425fb70daaf4f5e8bf84f5e5143e89f; no voters: 339f163711b143c6a6a210b47915fb59, 7702d30841eb48b79a91eaaeaf45e426
I20260806 09:32:14.203617 10972 raft_consensus.cc:2726] T bd5cd11c91154e56ad46d5768a63ba0e P 5425fb70daaf4f5e8bf84f5e5143e89f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 09:32:14.347249 10973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:14.346440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44265 (local address 127.20.113.3:59253)
0806 09:32:14.346970 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0806 09:32:14.347058 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:44265: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":309,"threads_started":1}
I20260806 09:32:14.405423 10974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:14.404442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44265 (local address 127.20.113.1:35093)
0806 09:32:14.405070 (+ 628us) negotiation.cc:107] Waiting for socket to connect
0806 09:32:14.405204 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:44265: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":509,"thread_start_us":279,"threads_started":1}
I20260806 09:32:14.881908 10975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:14.881120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44265 (local address 127.20.113.2:51279)
0806 09:32:14.881623 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0806 09:32:14.881711 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:44265: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":246,"threads_started":1}
I20260806 09:32:15.349017 10976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:15.348184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44265 (local address 127.20.113.3:57873)
0806 09:32:15.348749 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0806 09:32:15.348857 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:44265: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":306,"threads_started":1}
I20260806 09:32:15.407032 10977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:15.406338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44265 (local address 127.20.113.1:54411)
0806 09:32:15.406779 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0806 09:32:15.406879 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:44265: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":224,"threads_started":1}
I20260806 09:32:15.883862 10978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:15.882954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44265 (local address 127.20.113.2:42701)
0806 09:32:15.883545 (+ 591us) negotiation.cc:107] Waiting for socket to connect
0806 09:32:15.883645 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:44265: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":281,"threads_started":1}
I20260806 09:32:16.350674 10979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:16.349963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44265 (local address 127.20.113.3:54475)
0806 09:32:16.350407 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0806 09:32:16.350489 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:44265: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":237,"threads_started":1}
I20260806 09:32:16.409018 10980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:16.407993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44265 (local address 127.20.113.1:38373)
0806 09:32:16.408613 (+ 620us) negotiation.cc:107] Waiting for socket to connect
0806 09:32:16.408767 (+ 154us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:44265: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":358,"threads_started":1}
I20260806 09:32:16.629737 10982 raft_consensus.cc:482] T bd5cd11c91154e56ad46d5768a63ba0e P 5425fb70daaf4f5e8bf84f5e5143e89f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 09:32:16.629969 10982 raft_consensus.cc:504] T bd5cd11c91154e56ad46d5768a63ba0e P 5425fb70daaf4f5e8bf84f5e5143e89f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7702d30841eb48b79a91eaaeaf45e426" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 37455 } } peers { permanent_uuid: "5425fb70daaf4f5e8bf84f5e5143e89f" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 46039 } } peers { permanent_uuid: "339f163711b143c6a6a210b47915fb59" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 39849 } }
I20260806 09:32:16.630614 10982 leader_election.cc:288] T bd5cd11c91154e56ad46d5768a63ba0e P 5425fb70daaf4f5e8bf84f5e5143e89f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7702d30841eb48b79a91eaaeaf45e426 (127.20.113.2:37455), 339f163711b143c6a6a210b47915fb59 (127.20.113.3:39849)
W20260806 09:32:16.631705 10383 leader_election.cc:341] T bd5cd11c91154e56ad46d5768a63ba0e P 5425fb70daaf4f5e8bf84f5e5143e89f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7702d30841eb48b79a91eaaeaf45e426 (127.20.113.2:37455): Not found: Tablet not found: bd5cd11c91154e56ad46d5768a63ba0e
W20260806 09:32:16.631953 10386 leader_election.cc:341] T bd5cd11c91154e56ad46d5768a63ba0e P 5425fb70daaf4f5e8bf84f5e5143e89f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 339f163711b143c6a6a210b47915fb59 (127.20.113.3:39849): Not found: Tablet not found: bd5cd11c91154e56ad46d5768a63ba0e
I20260806 09:32:16.632109 10386 leader_election.cc:302] T bd5cd11c91154e56ad46d5768a63ba0e P 5425fb70daaf4f5e8bf84f5e5143e89f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5425fb70daaf4f5e8bf84f5e5143e89f; no voters: 339f163711b143c6a6a210b47915fb59, 7702d30841eb48b79a91eaaeaf45e426
I20260806 09:32:16.632426 10982 raft_consensus.cc:2726] T bd5cd11c91154e56ad46d5768a63ba0e P 5425fb70daaf4f5e8bf84f5e5143e89f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 09:32:16.885555 10983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:16.884813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44265 (local address 127.20.113.2:48591)
0806 09:32:16.885304 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0806 09:32:16.885391 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:44265: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":235,"threads_started":1}
I20260806 09:32:17.352383 10984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:17.351592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44265 (local address 127.20.113.3:54859)
0806 09:32:17.352079 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0806 09:32:17.352159 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:44265: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":385,"thread_start_us":255,"threads_started":1}
I20260806 09:32:17.410571 10985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:17.409909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44265 (local address 127.20.113.1:36397)
0806 09:32:17.410314 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0806 09:32:17.410413 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:44265: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":217,"threads_started":1}
I20260806 09:32:17.887120 10986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:17.886424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44265 (local address 127.20.113.2:49577)
0806 09:32:17.886870 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0806 09:32:17.886950 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:44265: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":231,"threads_started":1}
I20260806 09:32:18.354012 10987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:18.353265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44265 (local address 127.20.113.3:45289)
0806 09:32:18.353727 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0806 09:32:18.353827 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:44265: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":245,"threads_started":1}
I20260806 09:32:18.412526 10989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:18.411616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44265 (local address 127.20.113.1:50165)
0806 09:32:18.412137 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0806 09:32:18.412248 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:44265: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":416,"thread_start_us":285,"threads_started":1}
I20260806 09:32:18.888821 10990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:18.888057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44265 (local address 127.20.113.2:60963)
0806 09:32:18.888551 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0806 09:32:18.888649 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:44265: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":282,"threads_started":1}
I20260806 09:32:19.355710 10991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:19.354979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44265 (local address 127.20.113.3:39087)
0806 09:32:19.355453 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0806 09:32:19.355546 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:44265: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":257,"threads_started":1}
I20260806 09:32:19.414278 10992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:19.413513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44265 (local address 127.20.113.1:52037)
0806 09:32:19.414010 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0806 09:32:19.414093 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:44265: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":268,"threads_started":1}
I20260806 09:32:19.890707 10993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:19.889762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44265 (local address 127.20.113.2:36753)
0806 09:32:19.890278 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0806 09:32:19.890413 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:44265: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":288,"threads_started":1}
I20260806 09:32:20.357519 10994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:20.356767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44265 (local address 127.20.113.3:41129)
0806 09:32:20.357238 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0806 09:32:20.357327 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:44265: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":254,"threads_started":1}
I20260806 09:32:20.416241 10997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:20.415443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44265 (local address 127.20.113.1:33569)
0806 09:32:20.415987 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0806 09:32:20.416073 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:44265: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":297,"threads_started":1}
I20260806 09:32:20.892696 10998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:20.891783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44265 (local address 127.20.113.2:47251)
0806 09:32:20.892384 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0806 09:32:20.892524 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:44265: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":295,"threads_started":1}
I20260806 09:32:21.329746 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 10303
I20260806 09:32:21.351545 10999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:21.331042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37670 (local address 127.20.113.1:46039)
0806 09:32:21.331707 (+ 665us) server_negotiation.cc:206] Beginning negotiation
0806 09:32:21.331717 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 09:32:21.331754 (+ 37us) server_negotiation.cc:407] Connection header received
0806 09:32:21.331844 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:32:21.331854 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:32:21.332210 (+ 356us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:32:21.332443 (+ 233us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:32:21.334080 (+ 1637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:21.335126 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:21.336449 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:21.337038 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:21.337149 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:32:21.346269 (+ 9120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:32:21.346283 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:32:21.346298 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:32:21.346341 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:32:21.349084 (+ 2743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:32:21.349711 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:32:21.349718 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:32:21.349720 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:32:21.349855 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:32:21.350202 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:32:21.350206 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:32:21.350207 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:32:21.350566 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:32:21.350792 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0806 09:32:21.351053 (+ 261us) server_negotiation.cc:299] Negotiation successful
0806 09:32:21.351253 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":477,"thread_start_us":333,"threads_started":1}
I20260806 09:32:21.352829 10533 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:32:21.359596 11002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:21.358746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:44265 (local address 127.20.113.3:47993)
0806 09:32:21.359305 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0806 09:32:21.359409 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:44265: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":429,"thread_start_us":290,"threads_started":1}
I20260806 09:32:21.441143 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 10303
I20260806 09:32:21.470533 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 10547
I20260806 09:32:21.484532 11005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:21.471351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38780 (local address 127.20.113.2:37455)
0806 09:32:21.471838 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0806 09:32:21.471843 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:32:21.471858 (+ 15us) server_negotiation.cc:407] Connection header received
0806 09:32:21.471912 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:32:21.471915 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:32:21.471974 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:32:21.472096 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:32:21.473559 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:21.474310 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:21.475611 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:21.476058 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:21.476217 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:32:21.478922 (+ 2705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:32:21.478927 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:32:21.478930 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:32:21.478962 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:32:21.481653 (+ 2691us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:32:21.482418 (+ 765us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:32:21.482423 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:32:21.482426 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:32:21.482572 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:32:21.482982 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:32:21.482987 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:32:21.482990 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:32:21.483264 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:32:21.483577 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0806 09:32:21.483888 (+ 311us) server_negotiation.cc:299] Negotiation successful
0806 09:32:21.484133 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":371,"thread_start_us":239,"threads_started":1}
I20260806 09:32:21.485445 10716 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:32:21.569613 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 10547
I20260806 09:32:21.595966 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 10730
I20260806 09:32:21.610713 11009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:21.596954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32800 (local address 127.20.113.3:39849)
0806 09:32:21.597504 (+ 550us) server_negotiation.cc:206] Beginning negotiation
0806 09:32:21.597509 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:32:21.597527 (+ 18us) server_negotiation.cc:407] Connection header received
0806 09:32:21.597585 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:32:21.597588 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:32:21.597659 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:32:21.597753 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:32:21.599177 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:21.599988 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:21.601434 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:21.601889 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:21.601985 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:32:21.604782 (+ 2797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:32:21.604788 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:32:21.604799 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:32:21.604838 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:32:21.607788 (+ 2950us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:32:21.608706 (+ 918us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:32:21.608711 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:32:21.608714 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:32:21.608880 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:32:21.609274 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:32:21.609278 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:32:21.609280 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:32:21.609572 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:32:21.609801 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0806 09:32:21.610072 (+ 271us) server_negotiation.cc:299] Negotiation successful
0806 09:32:21.610253 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":430,"thread_start_us":289,"threads_started":1}
I20260806 09:32:21.611519 10911 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:32:21.699373 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 10730
2026-08-06T09:32:21Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (25024 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 09:32:21 dist-test-slave-45dx krb5kdc[11013](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 09:32:21 dist-test-slave-45dx krb5kdc[11013](info): set up 2 sockets
Aug 06 09:32:21 dist-test-slave-45dx krb5kdc[11013](info): commencing operation
krb5kdc: starting...
W20260806 09:32:23.038355 20932 mini_kdc.cc:121] Time spent starting KDC: real 1.279s user 0.004s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/krb5kdc/test-user.keytab.
Aug 06 09:32:23 dist-test-slave-45dx krb5kdc[11013](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008743, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T09:32:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T09:32:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.113.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.113.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.113.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab.
Entry for principal kudu/127.20.113.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab.
I20260806 09:32:23.365440 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.113.62:34891
--webserver_interface=127.20.113.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab
--principal=kudu/127.20.113.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:43715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.113.62:34891
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:32:23.708154 11028 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:32:23.708453 11028 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:32:23.708679 11028 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:32:23.715878 11028 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 09:32:23.715962 11028 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 09:32:23.716060 11028 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 09:32:23.716166 11028 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 09:32:23.716240 11028 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:32:23.716379 11028 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:32:23.716444 11028 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 09:32:23.716538 11028 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 09:32:23.726210 11028 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:43715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.20.113.62:34891
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab
--principal=kudu/127.20.113.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.113.62:34891
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.113.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:32:23.727483 11028 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:32:23.728924 11028 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:32:23.741133 11038 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:32:23.741632 11035 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:32:23.741086 11042 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:32:23.742143 11028 server_base.cc:1034] running on GCE node
I20260806 09:32:23.742864 11028 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:32:23.744145 11028 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:32:23.745373 11028 hybrid_clock.cc:648] HybridClock initialized: now 1786008743745331 us; error 56 us; skew 500 ppm
I20260806 09:32:23.745726 11028 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:32:23 dist-test-slave-45dx krb5kdc[11013](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008743, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:32:23.748979 11028 init.cc:377] Logged in from keytab as kudu/127.20.113.62@KRBTEST.COM (short username kudu)
I20260806 09:32:23.750481 11028 webserver.cc:460] Webserver started at http://127.20.113.62:38191/ using document root <none> and password file <none>
I20260806 09:32:23.751192 11028 fs_manager.cc:359] Metadata directory not provided
I20260806 09:32:23.751324 11028 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:32:23.751649 11028 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:32:23.754699 11028 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/master-0/data/instance:
uuid: "271408bd5ef14ab2a87473e851c18c92"
format_stamp: "Formatted at 2026-08-06 09:32:23 on dist-test-slave-45dx"
server_key: "999c3c72152d683e38f5ee2aff82de7e"
server_key_iv: "77dc09adab2edcd0fc586ffc80d5fa28"
server_key_version: "encryptionkey@0"
I20260806 09:32:23.755527 11028 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/master-0/wal/instance:
uuid: "271408bd5ef14ab2a87473e851c18c92"
format_stamp: "Formatted at 2026-08-06 09:32:23 on dist-test-slave-45dx"
server_key: "999c3c72152d683e38f5ee2aff82de7e"
server_key_iv: "77dc09adab2edcd0fc586ffc80d5fa28"
server_key_version: "encryptionkey@0"
I20260806 09:32:23.761657 11028 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260806 09:32:23.765182 11054 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:32:23.766489 11028 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 09:32:23.766677 11028 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/master-0/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/master-0/wal
uuid: "271408bd5ef14ab2a87473e851c18c92"
format_stamp: "Formatted at 2026-08-06 09:32:23 on dist-test-slave-45dx"
server_key: "999c3c72152d683e38f5ee2aff82de7e"
server_key_iv: "77dc09adab2edcd0fc586ffc80d5fa28"
server_key_version: "encryptionkey@0"
I20260806 09:32:23.766906 11028 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:32:23.789628 11028 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:32:23.793473 11028 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:32:23.793751 11028 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:32:23.812644 11028 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.62:34891
I20260806 09:32:23.812656 11122 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.62:34891 every 8 connection(s)
I20260806 09:32:23.813978 11028 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
I20260806 09:32:23.816250 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 11028
I20260806 09:32:23.816563 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/master-0/wal/instance
I20260806 09:32:23.817272 20932 external_mini_cluster.cc:1428] Setting key b3b616583f07421412dfc400d5a8f454
I20260806 09:32:23.817862 11124 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 06 09:32:23 dist-test-slave-45dx krb5kdc[11013](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008743, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.113.62@KRBTEST.COM
I20260806 09:32:23.832465 11124 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 271408bd5ef14ab2a87473e851c18c92: Bootstrap starting.
I20260806 09:32:23.836153 11124 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 271408bd5ef14ab2a87473e851c18c92: Neither blocks nor log segments found. Creating new log.
I20260806 09:32:23.837428 11124 log.cc:824] T 00000000000000000000000000000000 P 271408bd5ef14ab2a87473e851c18c92: Log is configured to *not* fsync() on all Append() calls
I20260806 09:32:23.840759 11124 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 271408bd5ef14ab2a87473e851c18c92: No bootstrap required, opened a new log
I20260806 09:32:23.841329 11130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:23.819267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58158 (local address 127.20.113.62:34891)
0806 09:32:23.820085 (+ 818us) server_negotiation.cc:206] Beginning negotiation
0806 09:32:23.820100 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 09:32:23.820151 (+ 51us) server_negotiation.cc:407] Connection header received
0806 09:32:23.821859 (+ 1708us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:32:23.821885 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:32:23.822366 (+ 481us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:32:23.822804 (+ 438us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:32:23.824192 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:23.826266 (+ 2074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:23.827615 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:23.828681 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:23.828880 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:32:23.833613 (+ 4733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:32:23.833621 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:32:23.833635 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:32:23.833684 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:32:23.837061 (+ 3377us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:32:23.837971 (+ 910us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:32:23.837985 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:32:23.837996 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:32:23.838259 (+ 263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:32:23.838650 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:32:23.838655 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:32:23.838659 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:32:23.839193 (+ 534us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:32:23.839413 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0806 09:32:23.839922 (+ 509us) server_negotiation.cc:299] Negotiation successful
0806 09:32:23.840257 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":580,"thread_start_us":333,"threads_started":1}
I20260806 09:32:23.848217 11124 raft_consensus.cc:348] T 00000000000000000000000000000000 P 271408bd5ef14ab2a87473e851c18c92 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "271408bd5ef14ab2a87473e851c18c92" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 34891 } }
I20260806 09:32:23.848651 11124 raft_consensus.cc:374] T 00000000000000000000000000000000 P 271408bd5ef14ab2a87473e851c18c92 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:32:23.848763 11124 raft_consensus.cc:729] T 00000000000000000000000000000000 P 271408bd5ef14ab2a87473e851c18c92 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 271408bd5ef14ab2a87473e851c18c92, State: Initialized, Role: FOLLOWER
I20260806 09:32:23.849301 11124 consensus_queue.cc:260] T 00000000000000000000000000000000 P 271408bd5ef14ab2a87473e851c18c92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "271408bd5ef14ab2a87473e851c18c92" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 34891 } }
I20260806 09:32:23.849540 11124 raft_consensus.cc:388] T 00000000000000000000000000000000 P 271408bd5ef14ab2a87473e851c18c92 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 09:32:23.849699 11124 raft_consensus.cc:482] T 00000000000000000000000000000000 P 271408bd5ef14ab2a87473e851c18c92 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 09:32:23.849859 11124 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 271408bd5ef14ab2a87473e851c18c92 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:32:23.852067 11124 raft_consensus.cc:504] T 00000000000000000000000000000000 P 271408bd5ef14ab2a87473e851c18c92 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "271408bd5ef14ab2a87473e851c18c92" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 34891 } }
I20260806 09:32:23.852588 11124 leader_election.cc:302] T 00000000000000000000000000000000 P 271408bd5ef14ab2a87473e851c18c92 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 271408bd5ef14ab2a87473e851c18c92; no voters:
I20260806 09:32:23.853153 11124 leader_election.cc:288] T 00000000000000000000000000000000 P 271408bd5ef14ab2a87473e851c18c92 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 09:32:23.853300 11132 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 271408bd5ef14ab2a87473e851c18c92 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:32:23.853981 11132 raft_consensus.cc:686] T 00000000000000000000000000000000 P 271408bd5ef14ab2a87473e851c18c92 [term 1 LEADER]: Becoming Leader. State: Replica: 271408bd5ef14ab2a87473e851c18c92, State: Running, Role: LEADER
I20260806 09:32:23.854524 11132 consensus_queue.cc:237] T 00000000000000000000000000000000 P 271408bd5ef14ab2a87473e851c18c92 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "271408bd5ef14ab2a87473e851c18c92" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 34891 } }
I20260806 09:32:23.855144 11124 sys_catalog.cc:564] T 00000000000000000000000000000000 P 271408bd5ef14ab2a87473e851c18c92 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 09:32:23.857775 11133 sys_catalog.cc:455] T 00000000000000000000000000000000 P 271408bd5ef14ab2a87473e851c18c92 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "271408bd5ef14ab2a87473e851c18c92" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "271408bd5ef14ab2a87473e851c18c92" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 34891 } } }
I20260806 09:32:23.858232 11133 sys_catalog.cc:458] T 00000000000000000000000000000000 P 271408bd5ef14ab2a87473e851c18c92 [sys.catalog]: This master's current role is: LEADER
I20260806 09:32:23.859304 11134 sys_catalog.cc:455] T 00000000000000000000000000000000 P 271408bd5ef14ab2a87473e851c18c92 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 271408bd5ef14ab2a87473e851c18c92. Latest consensus state: current_term: 1 leader_uuid: "271408bd5ef14ab2a87473e851c18c92" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "271408bd5ef14ab2a87473e851c18c92" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 34891 } } }
I20260806 09:32:23.859750 11134 sys_catalog.cc:458] T 00000000000000000000000000000000 P 271408bd5ef14ab2a87473e851c18c92 [sys.catalog]: This master's current role is: LEADER
I20260806 09:32:23.860646 11146 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 09:32:23.865708 11146 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 09:32:23.875171 11146 catalog_manager.cc:1347] Generated new cluster ID: a75a1d7b473849b59cfacd61a09e2517
I20260806 09:32:23.875278 11146 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 09:32:23.893811 11146 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 09:32:23.895533 11146 catalog_manager.cc:1504] Loading token signing keys...
I20260806 09:32:23.905177 11146 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 271408bd5ef14ab2a87473e851c18c92: Generated new TSK 0
I20260806 09:32:23.906095 11146 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.113.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.113.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.113.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.1.keytab.
Entry for principal kudu/127.20.113.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.1.keytab.
I20260806 09:32:23.999739 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.113.1:0
--local_ip_for_outbound_sockets=127.20.113.1
--webserver_interface=127.20.113.1
--webserver_port=0
--tserver_master_addrs=127.20.113.62:34891
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.1.keytab
--principal=kudu/127.20.113.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:43715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:32:24.361860 11164 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:32:24.362157 11164 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:32:24.362362 11164 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:32:24.369768 11164 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:32:24.369879 11164 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:32:24.370093 11164 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.113.1
I20260806 09:32:24.379902 11164 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:43715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.1.keytab
--principal=kudu/127.20.113.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.113.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.113.1
--webserver_port=0
--tserver_master_addrs=127.20.113.62:34891
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.113.1
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:32:24.381243 11164 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:32:24.382567 11164 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:32:24.395810 11174 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:32:24.395752 11175 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:32:24.395752 11177 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:32:24.396654 11164 server_base.cc:1034] running on GCE node
I20260806 09:32:24.397330 11164 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:32:24.398264 11164 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:32:24.399504 11164 hybrid_clock.cc:648] HybridClock initialized: now 1786008744399448 us; error 46 us; skew 500 ppm
I20260806 09:32:24.399825 11164 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:32:24 dist-test-slave-45dx krb5kdc[11013](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008744, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:32:24.403095 11164 init.cc:377] Logged in from keytab as kudu/127.20.113.1@KRBTEST.COM (short username kudu)
I20260806 09:32:24.404670 11164 webserver.cc:460] Webserver started at http://127.20.113.1:42921/ using document root <none> and password file <none>
I20260806 09:32:24.405395 11164 fs_manager.cc:359] Metadata directory not provided
I20260806 09:32:24.405552 11164 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:32:24.405880 11164 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:32:24.408975 11164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-0/data/instance:
uuid: "7a9ac4e4dc6d486eb682884bd5f5f027"
format_stamp: "Formatted at 2026-08-06 09:32:24 on dist-test-slave-45dx"
server_key: "3ca9f2f0d1c49526f2dbb3cc9f9b72ed"
server_key_iv: "e83f8335594e7b2813f037c2985581a7"
server_key_version: "encryptionkey@0"
I20260806 09:32:24.409751 11164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-0/wal/instance:
uuid: "7a9ac4e4dc6d486eb682884bd5f5f027"
format_stamp: "Formatted at 2026-08-06 09:32:24 on dist-test-slave-45dx"
server_key: "3ca9f2f0d1c49526f2dbb3cc9f9b72ed"
server_key_iv: "e83f8335594e7b2813f037c2985581a7"
server_key_version: "encryptionkey@0"
I20260806 09:32:24.414970 11164 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 09:32:24.418555 11188 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:32:24.419919 11164 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 09:32:24.420125 11164 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-0/wal
uuid: "7a9ac4e4dc6d486eb682884bd5f5f027"
format_stamp: "Formatted at 2026-08-06 09:32:24 on dist-test-slave-45dx"
server_key: "3ca9f2f0d1c49526f2dbb3cc9f9b72ed"
server_key_iv: "e83f8335594e7b2813f037c2985581a7"
server_key_version: "encryptionkey@0"
I20260806 09:32:24.420347 11164 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:32:24.443177 11164 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:32:24.447350 11164 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:32:24.447662 11164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:32:24.448690 11164 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:32:24.450093 11164 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:32:24.450205 11164 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:32:24.450366 11164 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:32:24.450449 11164 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:32:24.486006 11164 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.1:33093
I20260806 09:32:24.486068 11357 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.1:33093 every 8 connection(s)
I20260806 09:32:24.487435 11164 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-0/data/info.pb
I20260806 09:32:24.495049 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 11164
I20260806 09:32:24.495277 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-0/wal/instance
I20260806 09:32:24.495927 20932 external_mini_cluster.cc:1428] Setting key 1683d8dafbeebf0cd8f199e6b5b158c7
Aug 06 09:32:24 dist-test-slave-45dx krb5kdc[11013](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008744, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.1@KRBTEST.COM for kudu/127.20.113.62@KRBTEST.COM
I20260806 09:32:24.509349 11360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:24.490074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.1:42227 (local address 127.20.113.62:34891)
0806 09:32:24.490519 (+ 445us) server_negotiation.cc:206] Beginning negotiation
0806 09:32:24.490524 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:32:24.491528 (+ 1004us) server_negotiation.cc:407] Connection header received
0806 09:32:24.493484 (+ 1956us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:32:24.493489 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:32:24.493584 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:32:24.493711 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:32:24.495207 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:24.495965 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:24.497730 (+ 1765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:24.498294 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:24.498434 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:32:24.502107 (+ 3673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:32:24.502115 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:32:24.502121 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:32:24.502165 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:32:24.505803 (+ 3638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:32:24.506678 (+ 875us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:32:24.506687 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:32:24.506693 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:32:24.506857 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:32:24.507277 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:32:24.507285 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:32:24.507290 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:32:24.507702 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:32:24.507853 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0806 09:32:24.508614 (+ 761us) server_negotiation.cc:299] Negotiation successful
0806 09:32:24.508869 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":220,"threads_started":1}
I20260806 09:32:24.509981 11361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:24.490646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:34891 (local address 127.20.113.1:42227)
0806 09:32:24.491367 (+ 721us) negotiation.cc:107] Waiting for socket to connect
0806 09:32:24.491405 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 09:32:24.493209 (+ 1804us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:32:24.493956 (+ 747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:32:24.493963 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:32:24.494463 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:32:24.495018 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:32:24.495036 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:32:24.496167 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:32:24.496172 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:32:24.497562 (+ 1390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:32:24.497575 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:32:24.498500 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:32:24.498505 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:32:24.498702 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:32:24.499428 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:32:24.499451 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:32:24.501880 (+ 2429us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:32:24.506051 (+ 4171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:32:24.506058 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:32:24.506069 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:32:24.506518 (+ 449us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:32:24.506974 (+ 456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:32:24.506978 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:32:24.506980 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:32:24.507157 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:32:24.507820 (+ 663us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:32:24.507825 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:32:24.508145 (+ 320us) client_negotiation.cc:769] Sending connection context
0806 09:32:24.508460 (+ 315us) client_negotiation.cc:240] Negotiation successful
0806 09:32:24.508851 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":292,"threads_started":1}
I20260806 09:32:24.511565 11358 heartbeater.cc:347] Connected to a master server at 127.20.113.62:34891
I20260806 09:32:24.511885 11358 heartbeater.cc:464] Registering TS with master...
I20260806 09:32:24.512676 11358 heartbeater.cc:511] Master 127.20.113.62:34891 requested a full tablet report, sending...
I20260806 09:32:24.514768 11076 ts_manager.cc:194] Registered new tserver with Master: 7a9ac4e4dc6d486eb682884bd5f5f027 (127.20.113.1:33093)
I20260806 09:32:24.516197 11076 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.113.1@KRBTEST.COM'} at 127.20.113.1:42227
WARNING: no policy specified for kudu/127.20.113.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.113.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.113.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.2.keytab.
Entry for principal kudu/127.20.113.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.2.keytab.
I20260806 09:32:24.573740 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.113.2:0
--local_ip_for_outbound_sockets=127.20.113.2
--webserver_interface=127.20.113.2
--webserver_port=0
--tserver_master_addrs=127.20.113.62:34891
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.2.keytab
--principal=kudu/127.20.113.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:43715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:32:24.954702 11367 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:32:24.955049 11367 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:32:24.955158 11367 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:32:24.962935 11367 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:32:24.963035 11367 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:32:24.963234 11367 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.113.2
I20260806 09:32:24.972921 11367 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:43715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.2.keytab
--principal=kudu/127.20.113.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.113.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.113.2
--webserver_port=0
--tserver_master_addrs=127.20.113.62:34891
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.113.2
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:32:24.974251 11367 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:32:24.975608 11367 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:32:24.988933 11373 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:32:24.988962 11372 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:32:24.989027 11377 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:32:24.990288 11367 server_base.cc:1034] running on GCE node
I20260806 09:32:24.990943 11367 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:32:24.991882 11367 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:32:24.993091 11367 hybrid_clock.cc:648] HybridClock initialized: now 1786008744993045 us; error 33 us; skew 500 ppm
I20260806 09:32:24.993418 11367 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:32:24 dist-test-slave-45dx krb5kdc[11013](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008744, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:32:24.996886 11367 init.cc:377] Logged in from keytab as kudu/127.20.113.2@KRBTEST.COM (short username kudu)
I20260806 09:32:24.998330 11367 webserver.cc:460] Webserver started at http://127.20.113.2:34949/ using document root <none> and password file <none>
I20260806 09:32:24.999025 11367 fs_manager.cc:359] Metadata directory not provided
I20260806 09:32:24.999130 11367 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:32:24.999413 11367 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:32:25.002523 11367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-1/data/instance:
uuid: "71b724c2b8d64bbc9a06f26510d924f0"
format_stamp: "Formatted at 2026-08-06 09:32:24 on dist-test-slave-45dx"
server_key: "e2dfc7850b6c0395f8b6b9cb468d370d"
server_key_iv: "29b07e889b2b3e7e2421576eca4a105b"
server_key_version: "encryptionkey@0"
I20260806 09:32:25.003259 11367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-1/wal/instance:
uuid: "71b724c2b8d64bbc9a06f26510d924f0"
format_stamp: "Formatted at 2026-08-06 09:32:24 on dist-test-slave-45dx"
server_key: "e2dfc7850b6c0395f8b6b9cb468d370d"
server_key_iv: "29b07e889b2b3e7e2421576eca4a105b"
server_key_version: "encryptionkey@0"
I20260806 09:32:25.008615 11367 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260806 09:32:25.012092 11385 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:32:25.013447 11367 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 09:32:25.013725 11367 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-1/wal
uuid: "71b724c2b8d64bbc9a06f26510d924f0"
format_stamp: "Formatted at 2026-08-06 09:32:24 on dist-test-slave-45dx"
server_key: "e2dfc7850b6c0395f8b6b9cb468d370d"
server_key_iv: "29b07e889b2b3e7e2421576eca4a105b"
server_key_version: "encryptionkey@0"
I20260806 09:32:25.013945 11367 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:32:25.033318 11367 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:32:25.037374 11367 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:32:25.037655 11367 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:32:25.038589 11367 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:32:25.039896 11367 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:32:25.039969 11367 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:32:25.040148 11367 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:32:25.040199 11367 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:32:25.074221 11367 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.2:32951
I20260806 09:32:25.074270 11548 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.2:32951 every 8 connection(s)
I20260806 09:32:25.075538 11367 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-1/data/info.pb
I20260806 09:32:25.078246 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 11367
I20260806 09:32:25.078476 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-1/wal/instance
I20260806 09:32:25.079003 20932 external_mini_cluster.cc:1428] Setting key c8f5edaf214629bfd29c93e16ca71d27
Aug 06 09:32:25 dist-test-slave-45dx krb5kdc[11013](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008744, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.2@KRBTEST.COM for kudu/127.20.113.62@KRBTEST.COM
I20260806 09:32:25.097599 11554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:25.078187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.2:50435 (local address 127.20.113.62:34891)
0806 09:32:25.078670 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0806 09:32:25.078675 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:32:25.079890 (+ 1215us) server_negotiation.cc:407] Connection header received
0806 09:32:25.081939 (+ 2049us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:32:25.081947 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:32:25.082047 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:32:25.082169 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:32:25.083575 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:25.084638 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:25.086077 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:25.086872 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:25.087021 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:32:25.090768 (+ 3747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:32:25.090776 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:32:25.090784 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:32:25.090830 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:32:25.094549 (+ 3719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:32:25.095322 (+ 773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:32:25.095330 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:32:25.095336 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:32:25.095503 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:32:25.095903 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:32:25.095910 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:32:25.095916 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:32:25.096288 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:32:25.096491 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0806 09:32:25.097110 (+ 619us) server_negotiation.cc:299] Negotiation successful
0806 09:32:25.097328 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":260,"threads_started":1}
I20260806 09:32:25.099022 11555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:25.078872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:34891 (local address 127.20.113.2:50435)
0806 09:32:25.079686 (+ 814us) negotiation.cc:107] Waiting for socket to connect
0806 09:32:25.079746 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0806 09:32:25.081669 (+ 1923us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:32:25.082371 (+ 702us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:32:25.082378 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:32:25.082861 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:32:25.083409 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:32:25.083425 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:32:25.084778 (+ 1353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:32:25.084782 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:32:25.085931 (+ 1149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:32:25.085943 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:32:25.087125 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:32:25.087131 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:32:25.087346 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:32:25.088057 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:32:25.088097 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:32:25.090548 (+ 2451us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:32:25.094696 (+ 4148us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:32:25.094702 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:32:25.094714 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:32:25.095186 (+ 472us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:32:25.095615 (+ 429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:32:25.095618 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:32:25.095620 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:32:25.095794 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:32:25.096463 (+ 669us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:32:25.096469 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:32:25.096765 (+ 296us) client_negotiation.cc:769] Sending connection context
0806 09:32:25.096988 (+ 223us) client_negotiation.cc:240] Negotiation successful
0806 09:32:25.097512 (+ 524us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"thread_start_us":329,"threads_started":1}
I20260806 09:32:25.100594 11549 heartbeater.cc:347] Connected to a master server at 127.20.113.62:34891
I20260806 09:32:25.100941 11549 heartbeater.cc:464] Registering TS with master...
I20260806 09:32:25.101684 11549 heartbeater.cc:511] Master 127.20.113.62:34891 requested a full tablet report, sending...
I20260806 09:32:25.103238 11076 ts_manager.cc:194] Registered new tserver with Master: 71b724c2b8d64bbc9a06f26510d924f0 (127.20.113.2:32951)
I20260806 09:32:25.104058 11076 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.113.2@KRBTEST.COM'} at 127.20.113.2:50435
WARNING: no policy specified for kudu/127.20.113.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.113.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.113.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.3.keytab.
Entry for principal kudu/127.20.113.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.3.keytab.
I20260806 09:32:25.155318 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.113.3:0
--local_ip_for_outbound_sockets=127.20.113.3
--webserver_interface=127.20.113.3
--webserver_port=0
--tserver_master_addrs=127.20.113.62:34891
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.3.keytab
--principal=kudu/127.20.113.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:43715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 09:32:25.519349 11358 heartbeater.cc:503] Master 127.20.113.62:34891 was elected leader, sending a full tablet report...
W20260806 09:32:25.524338 11561 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:32:25.524713 11561 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:32:25.524887 11561 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:32:25.532083 11561 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:32:25.532238 11561 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:32:25.532440 11561 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.113.3
I20260806 09:32:25.542534 11561 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:43715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.3.keytab
--principal=kudu/127.20.113.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.113.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.113.3
--webserver_port=0
--tserver_master_addrs=127.20.113.62:34891
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.113.3
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:32:25.543932 11561 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:32:25.545392 11561 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:32:25.559348 11567 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:32:25.559810 11566 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:32:25.559350 11569 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:32:25.560698 11561 server_base.cc:1034] running on GCE node
I20260806 09:32:25.561466 11561 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:32:25.562527 11561 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:32:25.563740 11561 hybrid_clock.cc:648] HybridClock initialized: now 1786008745563682 us; error 54 us; skew 500 ppm
I20260806 09:32:25.564081 11561 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:32:25 dist-test-slave-45dx krb5kdc[11013](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008745, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:32:25.567436 11561 init.cc:377] Logged in from keytab as kudu/127.20.113.3@KRBTEST.COM (short username kudu)
I20260806 09:32:25.568907 11561 webserver.cc:460] Webserver started at http://127.20.113.3:42761/ using document root <none> and password file <none>
I20260806 09:32:25.569574 11561 fs_manager.cc:359] Metadata directory not provided
I20260806 09:32:25.569706 11561 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:32:25.569980 11561 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:32:25.573213 11561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-2/data/instance:
uuid: "4627ec4e33c14d7987cd73370bd2aad7"
format_stamp: "Formatted at 2026-08-06 09:32:25 on dist-test-slave-45dx"
server_key: "43304b5340218053fb813ac4183e8ed2"
server_key_iv: "0505e48553258cad4ce1b9dc362b8cdd"
server_key_version: "encryptionkey@0"
I20260806 09:32:25.573925 11561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-2/wal/instance:
uuid: "4627ec4e33c14d7987cd73370bd2aad7"
format_stamp: "Formatted at 2026-08-06 09:32:25 on dist-test-slave-45dx"
server_key: "43304b5340218053fb813ac4183e8ed2"
server_key_iv: "0505e48553258cad4ce1b9dc362b8cdd"
server_key_version: "encryptionkey@0"
I20260806 09:32:25.579175 11561 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260806 09:32:25.582649 11577 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:32:25.583971 11561 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 09:32:25.584169 11561 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-2/wal
uuid: "4627ec4e33c14d7987cd73370bd2aad7"
format_stamp: "Formatted at 2026-08-06 09:32:25 on dist-test-slave-45dx"
server_key: "43304b5340218053fb813ac4183e8ed2"
server_key_iv: "0505e48553258cad4ce1b9dc362b8cdd"
server_key_version: "encryptionkey@0"
I20260806 09:32:25.584424 11561 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:32:25.602975 11561 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:32:25.607125 11561 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:32:25.607503 11561 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:32:25.608559 11561 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:32:25.609915 11561 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:32:25.610015 11561 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:32:25.610162 11561 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:32:25.610252 11561 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:32:25.645243 11561 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.3:43133
I20260806 09:32:25.645292 11745 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.3:43133 every 8 connection(s)
I20260806 09:32:25.646579 11561 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-2/data/info.pb
I20260806 09:32:25.648564 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 11561
I20260806 09:32:25.648792 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-2/wal/instance
I20260806 09:32:25.649312 20932 external_mini_cluster.cc:1428] Setting key 691a61796a0baa79d1ab10ee3214a4f8
Aug 06 09:32:25 dist-test-slave-45dx krb5kdc[11013](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008745, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.3@KRBTEST.COM for kudu/127.20.113.62@KRBTEST.COM
I20260806 09:32:25.667483 11748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:25.649355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.3:44563 (local address 127.20.113.62:34891)
0806 09:32:25.649899 (+ 544us) server_negotiation.cc:206] Beginning negotiation
0806 09:32:25.649906 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:32:25.651028 (+ 1122us) server_negotiation.cc:407] Connection header received
0806 09:32:25.653319 (+ 2291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:32:25.653323 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:32:25.653403 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:32:25.653494 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:32:25.654955 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:25.655746 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:25.657319 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:25.657739 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:25.657829 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:32:25.661321 (+ 3492us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:32:25.661334 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:32:25.661343 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:32:25.661375 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:32:25.664268 (+ 2893us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:32:25.665144 (+ 876us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:32:25.665150 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:32:25.665153 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:32:25.665284 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:32:25.665738 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:32:25.665742 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:32:25.665746 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:32:25.666015 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:32:25.666183 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0806 09:32:25.666832 (+ 649us) server_negotiation.cc:299] Negotiation successful
0806 09:32:25.667061 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":410,"thread_start_us":265,"threads_started":1}
I20260806 09:32:25.668438 11750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:25.650012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:34891 (local address 127.20.113.3:44563)
0806 09:32:25.650838 (+ 826us) negotiation.cc:107] Waiting for socket to connect
0806 09:32:25.650897 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0806 09:32:25.653069 (+ 2172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:32:25.653704 (+ 635us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:32:25.653711 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:32:25.654248 (+ 537us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:32:25.654806 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:32:25.654824 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:32:25.655896 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:32:25.655900 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:32:25.657197 (+ 1297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:32:25.657209 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:32:25.657938 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:32:25.657943 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:32:25.658124 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:32:25.658733 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:32:25.658759 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:32:25.661121 (+ 2362us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:32:25.664438 (+ 3317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:32:25.664446 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:32:25.664461 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:32:25.665001 (+ 540us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:32:25.665402 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:32:25.665407 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:32:25.665411 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:32:25.665617 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:32:25.666152 (+ 535us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:32:25.666160 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:32:25.666487 (+ 327us) client_negotiation.cc:769] Sending connection context
0806 09:32:25.666745 (+ 258us) client_negotiation.cc:240] Negotiation successful
0806 09:32:25.667258 (+ 513us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":521,"thread_start_us":317,"threads_started":1}
I20260806 09:32:25.669689 11746 heartbeater.cc:347] Connected to a master server at 127.20.113.62:34891
I20260806 09:32:25.670006 11746 heartbeater.cc:464] Registering TS with master...
I20260806 09:32:25.670773 11746 heartbeater.cc:511] Master 127.20.113.62:34891 requested a full tablet report, sending...
I20260806 09:32:25.672168 11076 ts_manager.cc:194] Registered new tserver with Master: 4627ec4e33c14d7987cd73370bd2aad7 (127.20.113.3:43133)
I20260806 09:32:25.673009 11076 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.113.3@KRBTEST.COM'} at 127.20.113.3:44563
I20260806 09:32:25.677232 20932 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 09:32:25.695475 11748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:25.682576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58164 (local address 127.20.113.62:34891)
0806 09:32:25.682730 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0806 09:32:25.682735 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:32:25.683203 (+ 468us) server_negotiation.cc:407] Connection header received
0806 09:32:25.683344 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:32:25.683347 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:32:25.683406 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:32:25.683486 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:32:25.684983 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:25.685710 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:25.687168 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:25.687570 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:25.687663 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:32:25.690372 (+ 2709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:32:25.690376 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:32:25.690379 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:32:25.690411 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:32:25.693098 (+ 2687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:32:25.693790 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:32:25.693793 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:32:25.693796 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:32:25.693929 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:32:25.694284 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:32:25.694287 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:32:25.694289 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:32:25.694583 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:32:25.694767 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0806 09:32:25.695071 (+ 304us) server_negotiation.cc:299] Negotiation successful
0806 09:32:25.695237 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 09:32:25.699481 11076 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58164:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 09:32:25.706516 11076 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 09:32:25.732316 11764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:25.718867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.1:33093 (local address 127.0.0.1:38930)
0806 09:32:25.719844 (+ 977us) negotiation.cc:107] Waiting for socket to connect
0806 09:32:25.719904 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0806 09:32:25.720101 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:32:25.721516 (+ 1415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:32:25.721527 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:32:25.721570 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:32:25.722116 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:32:25.722132 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:32:25.725004 (+ 2872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:32:25.725012 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:32:25.728254 (+ 3242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:32:25.728274 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:32:25.730968 (+ 2694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:32:25.730978 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:32:25.731294 (+ 316us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:32:25.731379 (+ 85us) client_negotiation.cc:769] Sending connection context
0806 09:32:25.731523 (+ 144us) client_negotiation.cc:240] Negotiation successful
0806 09:32:25.731832 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":656,"thread_start_us":466,"threads_started":1}
I20260806 09:32:25.733743 11767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:25.719129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38930 (local address 127.20.113.1:33093)
0806 09:32:25.720818 (+ 1689us) server_negotiation.cc:206] Beginning negotiation
0806 09:32:25.720827 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 09:32:25.720854 (+ 27us) server_negotiation.cc:407] Connection header received
0806 09:32:25.720956 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:32:25.720963 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:32:25.721224 (+ 261us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:32:25.722408 (+ 1184us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:32:25.722745 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:25.724825 (+ 2080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:25.728558 (+ 3733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:25.730646 (+ 2088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:25.732866 (+ 2220us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:32:25.733155 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0806 09:32:25.733229 (+ 74us) server_negotiation.cc:299] Negotiation successful
0806 09:32:25.733401 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1487,"thread_start_us":1322,"threads_started":1}
I20260806 09:32:25.739324 11256 tablet_service.cc:1467] Processing CreateTablet for tablet 3943eedb21be489f8a5755a81fca5524 (DEFAULT_TABLE table=test-table [id=8ec511585fbf4884b27124d5f860951f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:32:25.741149 11256 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3943eedb21be489f8a5755a81fca5524. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:32:25.741891 11772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:25.721816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57888 (local address 127.20.113.3:43133)
0806 09:32:25.734192 (+ 12376us) server_negotiation.cc:206] Beginning negotiation
0806 09:32:25.734204 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 09:32:25.734246 (+ 42us) server_negotiation.cc:407] Connection header received
0806 09:32:25.734368 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:32:25.734377 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:32:25.734658 (+ 281us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:32:25.735425 (+ 767us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:32:25.735710 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:25.737269 (+ 1559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:25.739027 (+ 1758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:25.740568 (+ 1541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:25.740719 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:32:25.740960 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0806 09:32:25.741202 (+ 242us) server_negotiation.cc:299] Negotiation successful
0806 09:32:25.741403 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12132,"thread_start_us":12324,"threads_started":1}
I20260806 09:32:25.741873 11764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:25.721303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.3:43133 (local address 127.0.0.1:57888)
0806 09:32:25.732574 (+ 11271us) negotiation.cc:107] Waiting for socket to connect
0806 09:32:25.732590 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 09:32:25.732686 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:32:25.734901 (+ 2215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:32:25.734905 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:32:25.734931 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:32:25.735235 (+ 304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:32:25.735252 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:32:25.737431 (+ 2179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:32:25.737437 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:32:25.738847 (+ 1410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:32:25.738862 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:32:25.740754 (+ 1892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:32:25.740762 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:32:25.740958 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:32:25.740990 (+ 32us) client_negotiation.cc:769] Sending connection context
0806 09:32:25.741118 (+ 128us) client_negotiation.cc:240] Negotiation successful
0806 09:32:25.741452 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11168,"mutex_wait_us":161,"thread_start_us":13289,"threads_started":1}
I20260806 09:32:25.742511 11769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:25.719493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58440 (local address 127.20.113.2:32951)
0806 09:32:25.721227 (+ 1734us) server_negotiation.cc:206] Beginning negotiation
0806 09:32:25.721237 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 09:32:25.732370 (+ 11133us) server_negotiation.cc:407] Connection header received
0806 09:32:25.733189 (+ 819us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:32:25.733198 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:32:25.733480 (+ 282us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:32:25.733674 (+ 194us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:32:25.734906 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:25.736143 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:25.738682 (+ 2539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:25.740720 (+ 2038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:25.740876 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:32:25.741075 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0806 09:32:25.742023 (+ 948us) server_negotiation.cc:299] Negotiation successful
0806 09:32:25.742199 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1545,"thread_start_us":1441,"threads_started":1}
I20260806 09:32:25.743108 11768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:25.719307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.2:32951 (local address 127.0.0.1:58440)
0806 09:32:25.732195 (+ 12888us) negotiation.cc:107] Waiting for socket to connect
0806 09:32:25.732215 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 09:32:25.732953 (+ 738us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:32:25.734153 (+ 1200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:32:25.734160 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:32:25.734193 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:32:25.734642 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:32:25.734661 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:32:25.736364 (+ 1703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:32:25.736369 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:32:25.738499 (+ 2130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:32:25.738514 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:32:25.741142 (+ 2628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:32:25.741149 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:32:25.741549 (+ 400us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:32:25.741591 (+ 42us) client_negotiation.cc:769] Sending connection context
0806 09:32:25.741682 (+ 91us) client_negotiation.cc:240] Negotiation successful
0806 09:32:25.741844 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12756,"mutex_wait_us":61,"thread_start_us":13374,"threads_started":1}
I20260806 09:32:25.746523 11453 tablet_service.cc:1467] Processing CreateTablet for tablet 3943eedb21be489f8a5755a81fca5524 (DEFAULT_TABLE table=test-table [id=8ec511585fbf4884b27124d5f860951f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:32:25.746569 11646 tablet_service.cc:1467] Processing CreateTablet for tablet 3943eedb21be489f8a5755a81fca5524 (DEFAULT_TABLE table=test-table [id=8ec511585fbf4884b27124d5f860951f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:32:25.747857 11453 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3943eedb21be489f8a5755a81fca5524. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:32:25.748126 11646 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3943eedb21be489f8a5755a81fca5524. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:32:25.757011 11776 tablet_bootstrap.cc:492] T 3943eedb21be489f8a5755a81fca5524 P 7a9ac4e4dc6d486eb682884bd5f5f027: Bootstrap starting.
I20260806 09:32:25.760932 11776 tablet_bootstrap.cc:654] T 3943eedb21be489f8a5755a81fca5524 P 7a9ac4e4dc6d486eb682884bd5f5f027: Neither blocks nor log segments found. Creating new log.
I20260806 09:32:25.761418 11777 tablet_bootstrap.cc:492] T 3943eedb21be489f8a5755a81fca5524 P 71b724c2b8d64bbc9a06f26510d924f0: Bootstrap starting.
I20260806 09:32:25.762790 11778 tablet_bootstrap.cc:492] T 3943eedb21be489f8a5755a81fca5524 P 4627ec4e33c14d7987cd73370bd2aad7: Bootstrap starting.
I20260806 09:32:25.762848 11776 log.cc:824] T 3943eedb21be489f8a5755a81fca5524 P 7a9ac4e4dc6d486eb682884bd5f5f027: Log is configured to *not* fsync() on all Append() calls
I20260806 09:32:25.765846 11777 tablet_bootstrap.cc:654] T 3943eedb21be489f8a5755a81fca5524 P 71b724c2b8d64bbc9a06f26510d924f0: Neither blocks nor log segments found. Creating new log.
I20260806 09:32:25.766206 11776 tablet_bootstrap.cc:492] T 3943eedb21be489f8a5755a81fca5524 P 7a9ac4e4dc6d486eb682884bd5f5f027: No bootstrap required, opened a new log
I20260806 09:32:25.766204 11778 tablet_bootstrap.cc:654] T 3943eedb21be489f8a5755a81fca5524 P 4627ec4e33c14d7987cd73370bd2aad7: Neither blocks nor log segments found. Creating new log.
I20260806 09:32:25.766621 11776 ts_tablet_manager.cc:1397] T 3943eedb21be489f8a5755a81fca5524 P 7a9ac4e4dc6d486eb682884bd5f5f027: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.003s
I20260806 09:32:25.767177 11777 log.cc:824] T 3943eedb21be489f8a5755a81fca5524 P 71b724c2b8d64bbc9a06f26510d924f0: Log is configured to *not* fsync() on all Append() calls
I20260806 09:32:25.767449 11778 log.cc:824] T 3943eedb21be489f8a5755a81fca5524 P 4627ec4e33c14d7987cd73370bd2aad7: Log is configured to *not* fsync() on all Append() calls
I20260806 09:32:25.772259 11777 tablet_bootstrap.cc:492] T 3943eedb21be489f8a5755a81fca5524 P 71b724c2b8d64bbc9a06f26510d924f0: No bootstrap required, opened a new log
I20260806 09:32:25.772585 11777 ts_tablet_manager.cc:1397] T 3943eedb21be489f8a5755a81fca5524 P 71b724c2b8d64bbc9a06f26510d924f0: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.010s
I20260806 09:32:25.776695 11778 tablet_bootstrap.cc:492] T 3943eedb21be489f8a5755a81fca5524 P 4627ec4e33c14d7987cd73370bd2aad7: No bootstrap required, opened a new log
I20260806 09:32:25.776983 11778 ts_tablet_manager.cc:1397] T 3943eedb21be489f8a5755a81fca5524 P 4627ec4e33c14d7987cd73370bd2aad7: Time spent bootstrapping tablet: real 0.014s user 0.005s sys 0.008s
I20260806 09:32:25.779317 11776 raft_consensus.cc:348] T 3943eedb21be489f8a5755a81fca5524 P 7a9ac4e4dc6d486eb682884bd5f5f027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a9ac4e4dc6d486eb682884bd5f5f027" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33093 } } peers { permanent_uuid: "71b724c2b8d64bbc9a06f26510d924f0" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 32951 } } peers { permanent_uuid: "4627ec4e33c14d7987cd73370bd2aad7" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 43133 } }
I20260806 09:32:25.779819 11776 raft_consensus.cc:374] T 3943eedb21be489f8a5755a81fca5524 P 7a9ac4e4dc6d486eb682884bd5f5f027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:32:25.779937 11776 raft_consensus.cc:729] T 3943eedb21be489f8a5755a81fca5524 P 7a9ac4e4dc6d486eb682884bd5f5f027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a9ac4e4dc6d486eb682884bd5f5f027, State: Initialized, Role: FOLLOWER
I20260806 09:32:25.780177 11777 raft_consensus.cc:348] T 3943eedb21be489f8a5755a81fca5524 P 71b724c2b8d64bbc9a06f26510d924f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a9ac4e4dc6d486eb682884bd5f5f027" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33093 } } peers { permanent_uuid: "71b724c2b8d64bbc9a06f26510d924f0" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 32951 } } peers { permanent_uuid: "4627ec4e33c14d7987cd73370bd2aad7" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 43133 } }
I20260806 09:32:25.780733 11777 raft_consensus.cc:374] T 3943eedb21be489f8a5755a81fca5524 P 71b724c2b8d64bbc9a06f26510d924f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:32:25.780608 11776 consensus_queue.cc:260] T 3943eedb21be489f8a5755a81fca5524 P 7a9ac4e4dc6d486eb682884bd5f5f027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a9ac4e4dc6d486eb682884bd5f5f027" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33093 } } peers { permanent_uuid: "71b724c2b8d64bbc9a06f26510d924f0" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 32951 } } peers { permanent_uuid: "4627ec4e33c14d7987cd73370bd2aad7" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 43133 } }
I20260806 09:32:25.780872 11777 raft_consensus.cc:729] T 3943eedb21be489f8a5755a81fca5524 P 71b724c2b8d64bbc9a06f26510d924f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71b724c2b8d64bbc9a06f26510d924f0, State: Initialized, Role: FOLLOWER
I20260806 09:32:25.781481 11777 consensus_queue.cc:260] T 3943eedb21be489f8a5755a81fca5524 P 71b724c2b8d64bbc9a06f26510d924f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a9ac4e4dc6d486eb682884bd5f5f027" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33093 } } peers { permanent_uuid: "71b724c2b8d64bbc9a06f26510d924f0" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 32951 } } peers { permanent_uuid: "4627ec4e33c14d7987cd73370bd2aad7" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 43133 } }
I20260806 09:32:25.782403 11776 ts_tablet_manager.cc:1428] T 3943eedb21be489f8a5755a81fca5524 P 7a9ac4e4dc6d486eb682884bd5f5f027: Time spent starting tablet: real 0.016s user 0.014s sys 0.000s
I20260806 09:32:25.783197 11549 heartbeater.cc:503] Master 127.20.113.62:34891 was elected leader, sending a full tablet report...
I20260806 09:32:25.783944 11777 ts_tablet_manager.cc:1428] T 3943eedb21be489f8a5755a81fca5524 P 71b724c2b8d64bbc9a06f26510d924f0: Time spent starting tablet: real 0.011s user 0.008s sys 0.003s
I20260806 09:32:25.787540 11778 raft_consensus.cc:348] T 3943eedb21be489f8a5755a81fca5524 P 4627ec4e33c14d7987cd73370bd2aad7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a9ac4e4dc6d486eb682884bd5f5f027" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33093 } } peers { permanent_uuid: "71b724c2b8d64bbc9a06f26510d924f0" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 32951 } } peers { permanent_uuid: "4627ec4e33c14d7987cd73370bd2aad7" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 43133 } }
I20260806 09:32:25.787976 11778 raft_consensus.cc:374] T 3943eedb21be489f8a5755a81fca5524 P 4627ec4e33c14d7987cd73370bd2aad7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:32:25.788100 11778 raft_consensus.cc:729] T 3943eedb21be489f8a5755a81fca5524 P 4627ec4e33c14d7987cd73370bd2aad7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4627ec4e33c14d7987cd73370bd2aad7, State: Initialized, Role: FOLLOWER
I20260806 09:32:25.788741 11778 consensus_queue.cc:260] T 3943eedb21be489f8a5755a81fca5524 P 4627ec4e33c14d7987cd73370bd2aad7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a9ac4e4dc6d486eb682884bd5f5f027" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33093 } } peers { permanent_uuid: "71b724c2b8d64bbc9a06f26510d924f0" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 32951 } } peers { permanent_uuid: "4627ec4e33c14d7987cd73370bd2aad7" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 43133 } }
I20260806 09:32:25.789824 11746 heartbeater.cc:503] Master 127.20.113.62:34891 was elected leader, sending a full tablet report...
I20260806 09:32:25.790261 11778 ts_tablet_manager.cc:1428] T 3943eedb21be489f8a5755a81fca5524 P 4627ec4e33c14d7987cd73370bd2aad7: Time spent starting tablet: real 0.013s user 0.010s sys 0.003s
W20260806 09:32:25.827697 11551 tablet.cc:2369] T 3943eedb21be489f8a5755a81fca5524 P 71b724c2b8d64bbc9a06f26510d924f0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 09:32:25.845124 11785 raft_consensus.cc:482] T 3943eedb21be489f8a5755a81fca5524 P 71b724c2b8d64bbc9a06f26510d924f0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 09:32:25.845494 11785 raft_consensus.cc:504] T 3943eedb21be489f8a5755a81fca5524 P 71b724c2b8d64bbc9a06f26510d924f0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a9ac4e4dc6d486eb682884bd5f5f027" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33093 } } peers { permanent_uuid: "71b724c2b8d64bbc9a06f26510d924f0" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 32951 } } peers { permanent_uuid: "4627ec4e33c14d7987cd73370bd2aad7" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 43133 } }
I20260806 09:32:25.847304 11785 leader_election.cc:288] T 3943eedb21be489f8a5755a81fca5524 P 71b724c2b8d64bbc9a06f26510d924f0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7a9ac4e4dc6d486eb682884bd5f5f027 (127.20.113.1:33093), 4627ec4e33c14d7987cd73370bd2aad7 (127.20.113.3:43133)
I20260806 09:32:25.854516 11787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:25.847727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.1:33093 (local address 127.20.113.2:54097)
0806 09:32:25.848540 (+ 813us) negotiation.cc:107] Waiting for socket to connect
0806 09:32:25.848563 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 09:32:25.848716 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:32:25.849286 (+ 570us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:32:25.849293 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:32:25.849331 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:32:25.849797 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:32:25.849817 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:32:25.850823 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:32:25.850830 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:32:25.852420 (+ 1590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:32:25.852432 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:32:25.853628 (+ 1196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:32:25.853634 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:32:25.853815 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:32:25.853839 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 09:32:25.853960 (+ 121us) client_negotiation.cc:240] Negotiation successful
0806 09:32:25.854083 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":676,"mutex_wait_us":56,"thread_start_us":453,"threads_started":1}
I20260806 09:32:25.854816 11772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:25.848907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.2:41587 (local address 127.20.113.3:43133)
0806 09:32:25.849193 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0806 09:32:25.849199 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:32:25.849603 (+ 404us) server_negotiation.cc:407] Connection header received
0806 09:32:25.849775 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:32:25.849778 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:32:25.849839 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:32:25.849947 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:32:25.850491 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:25.851220 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:25.853004 (+ 1784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:25.853796 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:25.854001 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:32:25.854073 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 09:32:25.854435 (+ 362us) server_negotiation.cc:299] Negotiation successful
0806 09:32:25.854566 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260806 09:32:25.854919 11767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:25.847812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.2:54097 (local address 127.20.113.1:33093)
0806 09:32:25.848017 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0806 09:32:25.848021 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 09:32:25.848665 (+ 644us) server_negotiation.cc:407] Connection header received
0806 09:32:25.848864 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:32:25.848869 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:32:25.848952 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:32:25.849071 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:32:25.849964 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:25.850673 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:25.852574 (+ 1901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:25.853394 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:25.853598 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:32:25.853674 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 09:32:25.854258 (+ 584us) server_negotiation.cc:299] Negotiation successful
0806 09:32:25.854389 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 09:32:25.855520 11788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:25.848698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.3:43133 (local address 127.20.113.2:41587)
0806 09:32:25.849487 (+ 789us) negotiation.cc:107] Waiting for socket to connect
0806 09:32:25.849511 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 09:32:25.849650 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:32:25.849973 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:32:25.849977 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:32:25.850005 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:32:25.850316 (+ 311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:32:25.850324 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:32:25.851367 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:32:25.851373 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:32:25.852852 (+ 1479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:32:25.852865 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:32:25.854023 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:32:25.854030 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:32:25.854233 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:32:25.854263 (+ 30us) client_negotiation.cc:769] Sending connection context
0806 09:32:25.855012 (+ 749us) client_negotiation.cc:240] Negotiation successful
0806 09:32:25.855142 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":646,"mutex_wait_us":1,"thread_start_us":431,"threads_started":1}
I20260806 09:32:25.856086 11288 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3943eedb21be489f8a5755a81fca5524" candidate_uuid: "71b724c2b8d64bbc9a06f26510d924f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a9ac4e4dc6d486eb682884bd5f5f027" is_pre_election: true
I20260806 09:32:25.856714 11288 raft_consensus.cc:2454] T 3943eedb21be489f8a5755a81fca5524 P 7a9ac4e4dc6d486eb682884bd5f5f027 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 71b724c2b8d64bbc9a06f26510d924f0 in term 0.
I20260806 09:32:25.856630 11677 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3943eedb21be489f8a5755a81fca5524" candidate_uuid: "71b724c2b8d64bbc9a06f26510d924f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4627ec4e33c14d7987cd73370bd2aad7" is_pre_election: true
I20260806 09:32:25.857107 11677 raft_consensus.cc:2454] T 3943eedb21be489f8a5755a81fca5524 P 4627ec4e33c14d7987cd73370bd2aad7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 71b724c2b8d64bbc9a06f26510d924f0 in term 0.
I20260806 09:32:25.857429 11386 leader_election.cc:302] T 3943eedb21be489f8a5755a81fca5524 P 71b724c2b8d64bbc9a06f26510d924f0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 71b724c2b8d64bbc9a06f26510d924f0, 7a9ac4e4dc6d486eb682884bd5f5f027; no voters:
I20260806 09:32:25.857867 11785 raft_consensus.cc:2781] T 3943eedb21be489f8a5755a81fca5524 P 71b724c2b8d64bbc9a06f26510d924f0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 09:32:25.858021 11785 raft_consensus.cc:482] T 3943eedb21be489f8a5755a81fca5524 P 71b724c2b8d64bbc9a06f26510d924f0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 09:32:25.858129 11785 raft_consensus.cc:3037] T 3943eedb21be489f8a5755a81fca5524 P 71b724c2b8d64bbc9a06f26510d924f0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:32:25.860594 11785 raft_consensus.cc:504] T 3943eedb21be489f8a5755a81fca5524 P 71b724c2b8d64bbc9a06f26510d924f0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a9ac4e4dc6d486eb682884bd5f5f027" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33093 } } peers { permanent_uuid: "71b724c2b8d64bbc9a06f26510d924f0" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 32951 } } peers { permanent_uuid: "4627ec4e33c14d7987cd73370bd2aad7" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 43133 } }
I20260806 09:32:25.861296 11785 leader_election.cc:288] T 3943eedb21be489f8a5755a81fca5524 P 71b724c2b8d64bbc9a06f26510d924f0 [CANDIDATE]: Term 1 election: Requested vote from peers 7a9ac4e4dc6d486eb682884bd5f5f027 (127.20.113.1:33093), 4627ec4e33c14d7987cd73370bd2aad7 (127.20.113.3:43133)
I20260806 09:32:25.861680 11288 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3943eedb21be489f8a5755a81fca5524" candidate_uuid: "71b724c2b8d64bbc9a06f26510d924f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a9ac4e4dc6d486eb682884bd5f5f027"
I20260806 09:32:25.861819 11677 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3943eedb21be489f8a5755a81fca5524" candidate_uuid: "71b724c2b8d64bbc9a06f26510d924f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4627ec4e33c14d7987cd73370bd2aad7"
I20260806 09:32:25.861881 11288 raft_consensus.cc:3037] T 3943eedb21be489f8a5755a81fca5524 P 7a9ac4e4dc6d486eb682884bd5f5f027 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:32:25.862041 11677 raft_consensus.cc:3037] T 3943eedb21be489f8a5755a81fca5524 P 4627ec4e33c14d7987cd73370bd2aad7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:32:25.864107 11288 raft_consensus.cc:2454] T 3943eedb21be489f8a5755a81fca5524 P 7a9ac4e4dc6d486eb682884bd5f5f027 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 71b724c2b8d64bbc9a06f26510d924f0 in term 1.
I20260806 09:32:25.864270 11677 raft_consensus.cc:2454] T 3943eedb21be489f8a5755a81fca5524 P 4627ec4e33c14d7987cd73370bd2aad7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 71b724c2b8d64bbc9a06f26510d924f0 in term 1.
I20260806 09:32:25.864542 11386 leader_election.cc:302] T 3943eedb21be489f8a5755a81fca5524 P 71b724c2b8d64bbc9a06f26510d924f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 71b724c2b8d64bbc9a06f26510d924f0, 7a9ac4e4dc6d486eb682884bd5f5f027; no voters:
I20260806 09:32:25.864822 11785 raft_consensus.cc:2781] T 3943eedb21be489f8a5755a81fca5524 P 71b724c2b8d64bbc9a06f26510d924f0 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:32:25.865278 11785 raft_consensus.cc:686] T 3943eedb21be489f8a5755a81fca5524 P 71b724c2b8d64bbc9a06f26510d924f0 [term 1 LEADER]: Becoming Leader. State: Replica: 71b724c2b8d64bbc9a06f26510d924f0, State: Running, Role: LEADER
I20260806 09:32:25.865763 11785 consensus_queue.cc:237] T 3943eedb21be489f8a5755a81fca5524 P 71b724c2b8d64bbc9a06f26510d924f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a9ac4e4dc6d486eb682884bd5f5f027" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33093 } } peers { permanent_uuid: "71b724c2b8d64bbc9a06f26510d924f0" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 32951 } } peers { permanent_uuid: "4627ec4e33c14d7987cd73370bd2aad7" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 43133 } }
I20260806 09:32:25.870668 11076 catalog_manager.cc:5495] T 3943eedb21be489f8a5755a81fca5524 P 71b724c2b8d64bbc9a06f26510d924f0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 71b724c2b8d64bbc9a06f26510d924f0 (127.20.113.2). New cstate: current_term: 1 leader_uuid: "71b724c2b8d64bbc9a06f26510d924f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a9ac4e4dc6d486eb682884bd5f5f027" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33093 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "71b724c2b8d64bbc9a06f26510d924f0" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 32951 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4627ec4e33c14d7987cd73370bd2aad7" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 43133 } health_report { overall_health: UNKNOWN } } }
I20260806 09:32:25.896764 11769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:25.891583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58446 (local address 127.20.113.2:32951)
0806 09:32:25.891772 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0806 09:32:25.891777 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:32:25.891797 (+ 20us) server_negotiation.cc:407] Connection header received
0806 09:32:25.892023 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:32:25.892027 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:32:25.892120 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:32:25.892248 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 09:32:25.892655 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:25.893395 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:25.894773 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:25.895212 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:25.895308 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:32:25.895742 (+ 434us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 09:32:25.896228 (+ 486us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 09:32:25.896424 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0806 09:32:25.896466 (+ 42us) server_negotiation.cc:299] Negotiation successful
0806 09:32:25.896532 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
W20260806 09:32:25.898283 11747 tablet.cc:2369] T 3943eedb21be489f8a5755a81fca5524 P 4627ec4e33c14d7987cd73370bd2aad7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 09:32:25.905189 11288 raft_consensus.cc:1261] T 3943eedb21be489f8a5755a81fca5524 P 7a9ac4e4dc6d486eb682884bd5f5f027 [term 1 FOLLOWER]: Refusing update from remote peer 71b724c2b8d64bbc9a06f26510d924f0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 09:32:25.905190 11677 raft_consensus.cc:1261] T 3943eedb21be489f8a5755a81fca5524 P 4627ec4e33c14d7987cd73370bd2aad7 [term 1 FOLLOWER]: Refusing update from remote peer 71b724c2b8d64bbc9a06f26510d924f0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 09:32:25.906168 11789 consensus_queue.cc:1035] T 3943eedb21be489f8a5755a81fca5524 P 71b724c2b8d64bbc9a06f26510d924f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a9ac4e4dc6d486eb682884bd5f5f027" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 09:32:25.906530 11785 consensus_queue.cc:1035] T 3943eedb21be489f8a5755a81fca5524 P 71b724c2b8d64bbc9a06f26510d924f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4627ec4e33c14d7987cd73370bd2aad7" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 43133 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 09:32:25.917677 11793 mvcc.cc:205] Tried to move back new op lower bound from 7315491823216967680 to 7315491823070228480. Current Snapshot: MvccSnapshot[applied={T|T < 7315491823216967680}]
I20260806 09:32:25.919340 11794 mvcc.cc:205] Tried to move back new op lower bound from 7315491823216967680 to 7315491823070228480. Current Snapshot: MvccSnapshot[applied={T|T < 7315491823216967680}]
I20260806 09:32:25.931845 11075 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58164:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 09:32:25.932181 11075 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58164:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 09:32:25.935784 11075 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 271408bd5ef14ab2a87473e851c18c92: Sending DeleteTablet for 3 replicas of tablet 3943eedb21be489f8a5755a81fca5524
I20260806 09:32:25.936968 11453 tablet_service.cc:1514] Processing DeleteTablet for tablet 3943eedb21be489f8a5755a81fca5524 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:32:25 UTC) from {username='kudu', principal='kudu/127.20.113.62@KRBTEST.COM'} at 127.0.0.1:58440
I20260806 09:32:25.937167 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 11164
I20260806 09:32:25.937328 11646 tablet_service.cc:1514] Processing DeleteTablet for tablet 3943eedb21be489f8a5755a81fca5524 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:32:25 UTC) from {username='kudu', principal='kudu/127.20.113.62@KRBTEST.COM'} at 127.0.0.1:57888
I20260806 09:32:25.938464 11256 tablet_service.cc:1514] Processing DeleteTablet for tablet 3943eedb21be489f8a5755a81fca5524 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:32:25 UTC) from {username='kudu', principal='kudu/127.20.113.62@KRBTEST.COM'} at 127.0.0.1:38930
I20260806 09:32:25.953526 11810 tablet_replica.cc:330] T 3943eedb21be489f8a5755a81fca5524 P 71b724c2b8d64bbc9a06f26510d924f0: stopping tablet replica
I20260806 09:32:25.954147 11810 raft_consensus.cc:2229] T 3943eedb21be489f8a5755a81fca5524 P 71b724c2b8d64bbc9a06f26510d924f0 [term 1 LEADER]: Raft consensus shutting down.
I20260806 09:32:25.954835 11819 tablet_replica.cc:330] T 3943eedb21be489f8a5755a81fca5524 P 7a9ac4e4dc6d486eb682884bd5f5f027: stopping tablet replica
I20260806 09:32:25.955049 11810 raft_consensus.cc:2258] T 3943eedb21be489f8a5755a81fca5524 P 71b724c2b8d64bbc9a06f26510d924f0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 09:32:25.955480 11819 raft_consensus.cc:2229] T 3943eedb21be489f8a5755a81fca5524 P 7a9ac4e4dc6d486eb682884bd5f5f027 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 09:32:25.956127 11819 raft_consensus.cc:2258] T 3943eedb21be489f8a5755a81fca5524 P 7a9ac4e4dc6d486eb682884bd5f5f027 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 09:32:25.957015 11815 tablet_replica.cc:330] T 3943eedb21be489f8a5755a81fca5524 P 4627ec4e33c14d7987cd73370bd2aad7: stopping tablet replica
I20260806 09:32:25.957434 11815 raft_consensus.cc:2229] T 3943eedb21be489f8a5755a81fca5524 P 4627ec4e33c14d7987cd73370bd2aad7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 09:32:25.957927 11815 raft_consensus.cc:2258] T 3943eedb21be489f8a5755a81fca5524 P 4627ec4e33c14d7987cd73370bd2aad7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 09:32:25.960175 11815 ts_tablet_manager.cc:1905] T 3943eedb21be489f8a5755a81fca5524 P 4627ec4e33c14d7987cd73370bd2aad7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 09:32:25.960330 11819 ts_tablet_manager.cc:1905] T 3943eedb21be489f8a5755a81fca5524 P 7a9ac4e4dc6d486eb682884bd5f5f027: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 09:32:25.967649 11819 ts_tablet_manager.cc:1918] T 3943eedb21be489f8a5755a81fca5524 P 7a9ac4e4dc6d486eb682884bd5f5f027: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 09:32:25.967826 11819 log.cc:1196] T 3943eedb21be489f8a5755a81fca5524 P 7a9ac4e4dc6d486eb682884bd5f5f027: Deleting WAL directory at /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-0/wal/wals/3943eedb21be489f8a5755a81fca5524
I20260806 09:32:25.968130 11815 ts_tablet_manager.cc:1918] T 3943eedb21be489f8a5755a81fca5524 P 4627ec4e33c14d7987cd73370bd2aad7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 09:32:25.968411 11819 ts_tablet_manager.cc:1939] T 3943eedb21be489f8a5755a81fca5524 P 7a9ac4e4dc6d486eb682884bd5f5f027: Deleting consensus metadata
I20260806 09:32:25.968420 11815 log.cc:1196] T 3943eedb21be489f8a5755a81fca5524 P 4627ec4e33c14d7987cd73370bd2aad7: Deleting WAL directory at /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-2/wal/wals/3943eedb21be489f8a5755a81fca5524
I20260806 09:32:25.969059 11815 ts_tablet_manager.cc:1939] T 3943eedb21be489f8a5755a81fca5524 P 4627ec4e33c14d7987cd73370bd2aad7: Deleting consensus metadata
I20260806 09:32:25.968191 11810 ts_tablet_manager.cc:1905] T 3943eedb21be489f8a5755a81fca5524 P 71b724c2b8d64bbc9a06f26510d924f0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 09:32:25.970494 11055 catalog_manager.cc:4841] TS 7a9ac4e4dc6d486eb682884bd5f5f027 (127.20.113.1:33093): tablet 3943eedb21be489f8a5755a81fca5524 (table test-table [id=8ec511585fbf4884b27124d5f860951f]) successfully deleted
I20260806 09:32:25.971048 11055 catalog_manager.cc:4841] TS 4627ec4e33c14d7987cd73370bd2aad7 (127.20.113.3:43133): tablet 3943eedb21be489f8a5755a81fca5524 (table test-table [id=8ec511585fbf4884b27124d5f860951f]) successfully deleted
Aug 06 09:32:25 dist-test-slave-45dx krb5kdc[11013](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008743, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.113.1@KRBTEST.COM
I20260806 09:32:25.977681 11810 ts_tablet_manager.cc:1918] T 3943eedb21be489f8a5755a81fca5524 P 71b724c2b8d64bbc9a06f26510d924f0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 09:32:25.977888 11810 log.cc:1196] T 3943eedb21be489f8a5755a81fca5524 P 71b724c2b8d64bbc9a06f26510d924f0: Deleting WAL directory at /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-1/wal/wals/3943eedb21be489f8a5755a81fca5524
I20260806 09:32:25.978504 11810 ts_tablet_manager.cc:1939] T 3943eedb21be489f8a5755a81fca5524 P 71b724c2b8d64bbc9a06f26510d924f0: Deleting consensus metadata
I20260806 09:32:25.980346 11059 catalog_manager.cc:4841] TS 71b724c2b8d64bbc9a06f26510d924f0 (127.20.113.2:32951): tablet 3943eedb21be489f8a5755a81fca5524 (table test-table [id=8ec511585fbf4884b27124d5f860951f]) successfully deleted
I20260806 09:32:25.981803 11767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:25.938103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38946 (local address 127.20.113.1:33093)
0806 09:32:25.939121 (+ 1018us) server_negotiation.cc:206] Beginning negotiation
0806 09:32:25.939130 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 09:32:25.959255 (+ 20125us) server_negotiation.cc:407] Connection header received
0806 09:32:25.959612 (+ 357us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:32:25.959618 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:32:25.959722 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:32:25.959807 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:32:25.962312 (+ 2505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:25.963428 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:25.965509 (+ 2081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:25.966192 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:25.966292 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:32:25.974000 (+ 7708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:32:25.974015 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:32:25.974033 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:32:25.974088 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:32:25.978189 (+ 4101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:32:25.979185 (+ 996us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:32:25.979195 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:32:25.979198 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:32:25.979361 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:32:25.979871 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:32:25.979876 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:32:25.979879 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:32:25.980452 (+ 573us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:32:25.980834 (+ 382us) server_negotiation.cc:1035] Waiting for connection context
0806 09:32:25.981191 (+ 357us) server_negotiation.cc:299] Negotiation successful
0806 09:32:25.981417 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":851}
I20260806 09:32:25.982625 11350 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:32:26.073882 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 11164
I20260806 09:32:26.107760 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 11367
Aug 06 09:32:26 dist-test-slave-45dx krb5kdc[11013](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008743, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.113.2@KRBTEST.COM
I20260806 09:32:26.126530 11769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:26.108578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58452 (local address 127.20.113.2:32951)
0806 09:32:26.108918 (+ 340us) server_negotiation.cc:206] Beginning negotiation
0806 09:32:26.108925 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:32:26.108947 (+ 22us) server_negotiation.cc:407] Connection header received
0806 09:32:26.109115 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:32:26.109121 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:32:26.109207 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:32:26.109322 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:32:26.111064 (+ 1742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:26.112366 (+ 1302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:26.114234 (+ 1868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:26.114893 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:26.115021 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:32:26.120621 (+ 5600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:32:26.120632 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:32:26.120645 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:32:26.120679 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:32:26.123805 (+ 3126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:32:26.124598 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:32:26.124605 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:32:26.124608 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:32:26.124745 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:32:26.125152 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:32:26.125155 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:32:26.125157 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:32:26.125511 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:32:26.125748 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0806 09:32:26.126073 (+ 325us) server_negotiation.cc:299] Negotiation successful
0806 09:32:26.126226 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":194}
I20260806 09:32:26.127300 11543 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:32:26.242127 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 11367
I20260806 09:32:26.271569 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 11561
Aug 06 09:32:26 dist-test-slave-45dx krb5kdc[11013](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008743, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.113.3@KRBTEST.COM
I20260806 09:32:26.287297 11772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:26.272351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57902 (local address 127.20.113.3:43133)
0806 09:32:26.272532 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0806 09:32:26.272539 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:32:26.272608 (+ 69us) server_negotiation.cc:407] Connection header received
0806 09:32:26.272842 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:32:26.272849 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:32:26.272934 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:32:26.273051 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:32:26.274368 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:26.275057 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:26.276366 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:26.276771 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:26.276914 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:32:26.281675 (+ 4761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:32:26.281690 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:32:26.281707 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:32:26.281742 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:32:26.284521 (+ 2779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:32:26.285179 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:32:26.285187 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:32:26.285189 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:32:26.285336 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:32:26.285748 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:32:26.285753 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:32:26.285756 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:32:26.286119 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:32:26.286414 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0806 09:32:26.286753 (+ 339us) server_negotiation.cc:299] Negotiation successful
0806 09:32:26.286991 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 09:32:26.287957 11739 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:32:26.378146 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 11561
I20260806 09:32:26.403587 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 11028
I20260806 09:32:26.404742 11116 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:32:26.514647 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 11028
I20260806 09:32:46.545845 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.113.62:34891
--webserver_interface=127.20.113.62
--webserver_port=38191
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab
--principal=kudu/127.20.113.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:43715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.113.62:34891
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:32:46.903193 11824 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:32:46.903481 11824 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:32:46.903569 11824 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:32:46.910403 11824 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 09:32:46.910488 11824 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 09:32:46.910526 11824 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 09:32:46.910580 11824 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 09:32:46.910632 11824 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:32:46.910691 11824 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:32:46.910735 11824 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 09:32:46.910781 11824 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 09:32:46.920027 11824 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:43715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.20.113.62:34891
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab
--principal=kudu/127.20.113.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.113.62:34891
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.113.62
--webserver_port=38191
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:32:46.921267 11824 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:32:46.922613 11824 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:32:46.949259 11833 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:32:46.949221 11836 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:32:46.949184 11834 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:32:46.954046 11824 server_base.cc:1034] running on GCE node
I20260806 09:32:46.955063 11824 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:32:46.956849 11824 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:32:46.958171 11824 hybrid_clock.cc:648] HybridClock initialized: now 1786008766958090 us; error 45 us; skew 500 ppm
I20260806 09:32:46.958626 11824 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:32:46 dist-test-slave-45dx krb5kdc[11013](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008766, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:32:46.962551 11824 init.cc:377] Logged in from keytab as kudu/127.20.113.62@KRBTEST.COM (short username kudu)
I20260806 09:32:46.964470 11824 webserver.cc:460] Webserver started at http://127.20.113.62:38191/ using document root <none> and password file <none>
I20260806 09:32:46.965324 11824 fs_manager.cc:359] Metadata directory not provided
I20260806 09:32:46.965497 11824 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:32:46.971926 11824 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 09:32:46.975329 11847 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:32:46.976923 11824 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 09:32:46.977123 11824 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/master-0/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/master-0/wal
uuid: "271408bd5ef14ab2a87473e851c18c92"
format_stamp: "Formatted at 2026-08-06 09:32:23 on dist-test-slave-45dx"
server_key: "999c3c72152d683e38f5ee2aff82de7e"
server_key_iv: "77dc09adab2edcd0fc586ffc80d5fa28"
server_key_version: "encryptionkey@0"
I20260806 09:32:46.978241 11824 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:32:46.993567 11824 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:32:46.997951 11824 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:32:46.998281 11824 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:32:47.018862 11824 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.62:34891
I20260806 09:32:47.018916 11911 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.62:34891 every 8 connection(s)
I20260806 09:32:47.020610 11824 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
I20260806 09:32:47.024909 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 11824
I20260806 09:32:47.026005 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.113.1:33093
--local_ip_for_outbound_sockets=127.20.113.1
--tserver_master_addrs=127.20.113.62:34891
--webserver_port=42921
--webserver_interface=127.20.113.1
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.1.keytab
--principal=kudu/127.20.113.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:43715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 09:32:47.029623 11912 sys_catalog.cc:263] Verifying existing consensus state
I20260806 09:32:47.034998 11912 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 271408bd5ef14ab2a87473e851c18c92: Bootstrap starting.
I20260806 09:32:47.057950 11912 log.cc:824] T 00000000000000000000000000000000 P 271408bd5ef14ab2a87473e851c18c92: Log is configured to *not* fsync() on all Append() calls
I20260806 09:32:47.070344 11912 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 271408bd5ef14ab2a87473e851c18c92: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 09:32:47.070880 11912 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 271408bd5ef14ab2a87473e851c18c92: Bootstrap complete.
I20260806 09:32:47.080583 11912 raft_consensus.cc:348] T 00000000000000000000000000000000 P 271408bd5ef14ab2a87473e851c18c92 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "271408bd5ef14ab2a87473e851c18c92" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 34891 } }
I20260806 09:32:47.081449 11912 raft_consensus.cc:729] T 00000000000000000000000000000000 P 271408bd5ef14ab2a87473e851c18c92 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 271408bd5ef14ab2a87473e851c18c92, State: Initialized, Role: FOLLOWER
I20260806 09:32:47.082154 11912 consensus_queue.cc:260] T 00000000000000000000000000000000 P 271408bd5ef14ab2a87473e851c18c92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "271408bd5ef14ab2a87473e851c18c92" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 34891 } }
I20260806 09:32:47.082436 11912 raft_consensus.cc:388] T 00000000000000000000000000000000 P 271408bd5ef14ab2a87473e851c18c92 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 09:32:47.082584 11912 raft_consensus.cc:482] T 00000000000000000000000000000000 P 271408bd5ef14ab2a87473e851c18c92 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 09:32:47.082794 11912 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 271408bd5ef14ab2a87473e851c18c92 [term 1 FOLLOWER]: Advancing to term 2
I20260806 09:32:47.086934 11912 raft_consensus.cc:504] T 00000000000000000000000000000000 P 271408bd5ef14ab2a87473e851c18c92 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "271408bd5ef14ab2a87473e851c18c92" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 34891 } }
I20260806 09:32:47.087476 11912 leader_election.cc:302] T 00000000000000000000000000000000 P 271408bd5ef14ab2a87473e851c18c92 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 271408bd5ef14ab2a87473e851c18c92; no voters:
I20260806 09:32:47.088157 11912 leader_election.cc:288] T 00000000000000000000000000000000 P 271408bd5ef14ab2a87473e851c18c92 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 09:32:47.088294 11918 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 271408bd5ef14ab2a87473e851c18c92 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 09:32:47.089679 11918 raft_consensus.cc:686] T 00000000000000000000000000000000 P 271408bd5ef14ab2a87473e851c18c92 [term 2 LEADER]: Becoming Leader. State: Replica: 271408bd5ef14ab2a87473e851c18c92, State: Running, Role: LEADER
I20260806 09:32:47.090296 11918 consensus_queue.cc:237] T 00000000000000000000000000000000 P 271408bd5ef14ab2a87473e851c18c92 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "271408bd5ef14ab2a87473e851c18c92" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 34891 } }
I20260806 09:32:47.090559 11912 sys_catalog.cc:564] T 00000000000000000000000000000000 P 271408bd5ef14ab2a87473e851c18c92 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 09:32:47.092911 11922 sys_catalog.cc:455] T 00000000000000000000000000000000 P 271408bd5ef14ab2a87473e851c18c92 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 271408bd5ef14ab2a87473e851c18c92. Latest consensus state: current_term: 2 leader_uuid: "271408bd5ef14ab2a87473e851c18c92" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "271408bd5ef14ab2a87473e851c18c92" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 34891 } } }
I20260806 09:32:47.093298 11922 sys_catalog.cc:458] T 00000000000000000000000000000000 P 271408bd5ef14ab2a87473e851c18c92 [sys.catalog]: This master's current role is: LEADER
I20260806 09:32:47.093878 11920 sys_catalog.cc:455] T 00000000000000000000000000000000 P 271408bd5ef14ab2a87473e851c18c92 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "271408bd5ef14ab2a87473e851c18c92" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "271408bd5ef14ab2a87473e851c18c92" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 34891 } } }
I20260806 09:32:47.094293 11920 sys_catalog.cc:458] T 00000000000000000000000000000000 P 271408bd5ef14ab2a87473e851c18c92 [sys.catalog]: This master's current role is: LEADER
I20260806 09:32:47.097638 11929 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 09:32:47.108896 11929 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 09:32:47.110388 11929 catalog_manager.cc:1259] Loaded cluster ID: a75a1d7b473849b59cfacd61a09e2517
I20260806 09:32:47.110558 11929 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 09:32:47.114873 11929 catalog_manager.cc:1504] Loading token signing keys...
I20260806 09:32:47.116601 11929 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 271408bd5ef14ab2a87473e851c18c92: Loaded TSK: 0
I20260806 09:32:47.162498 11929 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 271408bd5ef14ab2a87473e851c18c92: Generated new TSK 1
I20260806 09:32:47.162977 11929 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 09:32:47.407385 11915 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:32:47.407776 11915 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:32:47.407950 11915 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:32:47.419536 11915 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:32:47.419682 11915 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:32:47.419911 11915 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.113.1
I20260806 09:32:47.436108 11915 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:43715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.1.keytab
--principal=kudu/127.20.113.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.113.1:33093
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.113.1
--webserver_port=42921
--tserver_master_addrs=127.20.113.62:34891
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.113.1
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:32:47.437945 11915 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:32:47.439867 11915 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:32:47.456367 11951 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:32:47.456689 11954 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:32:47.456780 11950 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:32:47.457528 11915 server_base.cc:1034] running on GCE node
I20260806 09:32:47.458189 11915 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:32:47.459240 11915 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:32:47.460493 11915 hybrid_clock.cc:648] HybridClock initialized: now 1786008767460432 us; error 54 us; skew 500 ppm
I20260806 09:32:47.460829 11915 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:32:47 dist-test-slave-45dx krb5kdc[11013](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008767, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:32:47.468715 11915 init.cc:377] Logged in from keytab as kudu/127.20.113.1@KRBTEST.COM (short username kudu)
I20260806 09:32:47.470345 11915 webserver.cc:460] Webserver started at http://127.20.113.1:42921/ using document root <none> and password file <none>
I20260806 09:32:47.471067 11915 fs_manager.cc:359] Metadata directory not provided
I20260806 09:32:47.471226 11915 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:32:47.477165 11915 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 09:32:47.480283 11969 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:32:47.481994 11915 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 09:32:47.482185 11915 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-0/wal
uuid: "7a9ac4e4dc6d486eb682884bd5f5f027"
format_stamp: "Formatted at 2026-08-06 09:32:24 on dist-test-slave-45dx"
server_key: "3ca9f2f0d1c49526f2dbb3cc9f9b72ed"
server_key_iv: "e83f8335594e7b2813f037c2985581a7"
server_key_version: "encryptionkey@0"
I20260806 09:32:47.483228 11915 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:32:47.518533 11915 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:32:47.522989 11915 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:32:47.523275 11915 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:32:47.524257 11915 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:32:47.525633 11915 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:32:47.525758 11915 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:32:47.525899 11915 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:32:47.525990 11915 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:32:47.560034 11915 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.1:33093
I20260806 09:32:47.560083 12132 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.1:33093 every 8 connection(s)
I20260806 09:32:47.561618 11915 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-0/data/info.pb
I20260806 09:32:47.563529 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 11915
I20260806 09:32:47.564302 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.113.2:32951
--local_ip_for_outbound_sockets=127.20.113.2
--tserver_master_addrs=127.20.113.62:34891
--webserver_port=34949
--webserver_interface=127.20.113.2
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.2.keytab
--principal=kudu/127.20.113.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:43715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 09:32:47 dist-test-slave-45dx krb5kdc[11013](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008767, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.1@KRBTEST.COM for kudu/127.20.113.62@KRBTEST.COM
I20260806 09:32:47.587090 12136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:47.564641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.1:42765 (local address 127.20.113.62:34891)
0806 09:32:47.565274 (+ 633us) server_negotiation.cc:206] Beginning negotiation
0806 09:32:47.565291 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 09:32:47.565921 (+ 630us) server_negotiation.cc:407] Connection header received
0806 09:32:47.568107 (+ 2186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:32:47.568127 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:32:47.568611 (+ 484us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:32:47.569024 (+ 413us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:32:47.570366 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:47.572220 (+ 1854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:47.573839 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:47.574873 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:47.575065 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:32:47.578737 (+ 3672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:32:47.578748 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:32:47.578766 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:32:47.578818 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:32:47.582462 (+ 3644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:32:47.583409 (+ 947us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:32:47.583417 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:32:47.583425 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:32:47.583662 (+ 237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:32:47.584168 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:32:47.584172 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:32:47.584174 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:32:47.584690 (+ 516us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:32:47.584833 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0806 09:32:47.585813 (+ 980us) server_negotiation.cc:299] Negotiation successful
0806 09:32:47.586109 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":255,"threads_started":1}
I20260806 09:32:47.587603 12137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:47.564813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:34891 (local address 127.20.113.1:42765)
0806 09:32:47.565717 (+ 904us) negotiation.cc:107] Waiting for socket to connect
0806 09:32:47.565768 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 09:32:47.567517 (+ 1749us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:32:47.569135 (+ 1618us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:32:47.569142 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:32:47.569654 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:32:47.570154 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:32:47.570170 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:32:47.572467 (+ 2297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:32:47.572471 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:32:47.573681 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:32:47.573692 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:32:47.575092 (+ 1400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:32:47.575096 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:32:47.575276 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:32:47.575982 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:32:47.576008 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:32:47.578473 (+ 2465us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:32:47.582605 (+ 4132us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:32:47.582616 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:32:47.582632 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:32:47.583250 (+ 618us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:32:47.583796 (+ 546us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:32:47.583804 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:32:47.583810 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:32:47.584052 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:32:47.584842 (+ 790us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:32:47.584852 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:32:47.585330 (+ 478us) client_negotiation.cc:769] Sending connection context
0806 09:32:47.585659 (+ 329us) client_negotiation.cc:240] Negotiation successful
0806 09:32:47.586200 (+ 541us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":609,"thread_start_us":362,"threads_started":1}
I20260806 09:32:47.590823 12133 heartbeater.cc:347] Connected to a master server at 127.20.113.62:34891
I20260806 09:32:47.591137 12133 heartbeater.cc:464] Registering TS with master...
I20260806 09:32:47.592082 12133 heartbeater.cc:511] Master 127.20.113.62:34891 requested a full tablet report, sending...
I20260806 09:32:47.594594 11869 ts_manager.cc:194] Registered new tserver with Master: 7a9ac4e4dc6d486eb682884bd5f5f027 (127.20.113.1:33093)
I20260806 09:32:47.596926 11869 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.113.1@KRBTEST.COM'} at 127.20.113.1:42765
W20260806 09:32:47.915442 12139 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:32:47.915706 12139 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:32:47.915864 12139 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:32:47.923002 12139 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:32:47.923106 12139 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:32:47.923283 12139 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.113.2
I20260806 09:32:47.932698 12139 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:43715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.2.keytab
--principal=kudu/127.20.113.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.113.2:32951
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.113.2
--webserver_port=34949
--tserver_master_addrs=127.20.113.62:34891
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.113.2
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:32:47.933959 12139 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:32:47.935274 12139 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:32:47.948357 12145 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:32:47.948414 12146 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:32:47.948522 12149 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:32:47.949359 12139 server_base.cc:1034] running on GCE node
I20260806 09:32:47.949985 12139 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:32:47.950888 12139 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:32:47.952116 12139 hybrid_clock.cc:648] HybridClock initialized: now 1786008767952079 us; error 52 us; skew 500 ppm
I20260806 09:32:47.952504 12139 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:32:47 dist-test-slave-45dx krb5kdc[11013](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008767, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:32:47.955525 12139 init.cc:377] Logged in from keytab as kudu/127.20.113.2@KRBTEST.COM (short username kudu)
I20260806 09:32:47.957108 12139 webserver.cc:460] Webserver started at http://127.20.113.2:34949/ using document root <none> and password file <none>
I20260806 09:32:47.957837 12139 fs_manager.cc:359] Metadata directory not provided
I20260806 09:32:47.958014 12139 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:32:47.963500 12139 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 09:32:47.966360 12161 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:32:47.967624 12139 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 09:32:47.967803 12139 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-1/wal
uuid: "71b724c2b8d64bbc9a06f26510d924f0"
format_stamp: "Formatted at 2026-08-06 09:32:24 on dist-test-slave-45dx"
server_key: "e2dfc7850b6c0395f8b6b9cb468d370d"
server_key_iv: "29b07e889b2b3e7e2421576eca4a105b"
server_key_version: "encryptionkey@0"
I20260806 09:32:47.968834 12139 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:32:47.993021 12139 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:32:47.998142 12139 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:32:47.998459 12139 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:32:47.999550 12139 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:32:48.000938 12139 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:32:48.001061 12139 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:32:48.001186 12139 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:32:48.001286 12139 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:32:48.036970 12139 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.2:32951
I20260806 09:32:48.037020 12328 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.2:32951 every 8 connection(s)
I20260806 09:32:48.038332 12139 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-1/data/info.pb
I20260806 09:32:48.043177 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 12139
I20260806 09:32:48.043814 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.113.3:43133
--local_ip_for_outbound_sockets=127.20.113.3
--tserver_master_addrs=127.20.113.62:34891
--webserver_port=42761
--webserver_interface=127.20.113.3
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.3.keytab
--principal=kudu/127.20.113.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:43715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 09:32:48 dist-test-slave-45dx krb5kdc[11013](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008767, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.2@KRBTEST.COM for kudu/127.20.113.62@KRBTEST.COM
I20260806 09:32:48.059202 12136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:48.040963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.2:58417 (local address 127.20.113.62:34891)
0806 09:32:48.041156 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0806 09:32:48.041160 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 09:32:48.042263 (+ 1103us) server_negotiation.cc:407] Connection header received
0806 09:32:48.044378 (+ 2115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:32:48.044386 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:32:48.044482 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:32:48.044591 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:32:48.046038 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:48.046719 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:48.048167 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:48.048649 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:48.048779 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:32:48.053071 (+ 4292us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:32:48.053077 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:32:48.053080 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:32:48.053112 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:32:48.055712 (+ 2600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:32:48.056653 (+ 941us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:32:48.056657 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:32:48.056659 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:32:48.056779 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:32:48.057255 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:32:48.057258 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:32:48.057260 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:32:48.057503 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:32:48.057612 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 09:32:48.058512 (+ 900us) server_negotiation.cc:299] Negotiation successful
0806 09:32:48.058772 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 09:32:48.059917 12332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:48.041379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:34891 (local address 127.20.113.2:58417)
0806 09:32:48.042095 (+ 716us) negotiation.cc:107] Waiting for socket to connect
0806 09:32:48.042141 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 09:32:48.044095 (+ 1954us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:32:48.044825 (+ 730us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:32:48.044833 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:32:48.045323 (+ 490us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:32:48.045867 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:32:48.045885 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:32:48.046878 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:32:48.046888 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:32:48.048013 (+ 1125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:32:48.048024 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:32:48.048886 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:32:48.048894 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:32:48.049113 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:32:48.049837 (+ 724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:32:48.049879 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:32:48.052825 (+ 2946us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:32:48.055867 (+ 3042us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:32:48.055885 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:32:48.055901 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:32:48.056544 (+ 643us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:32:48.056895 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:32:48.056903 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:32:48.056909 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:32:48.057150 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:32:48.057626 (+ 476us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:32:48.057636 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:32:48.058058 (+ 422us) client_negotiation.cc:769] Sending connection context
0806 09:32:48.058398 (+ 340us) client_negotiation.cc:240] Negotiation successful
0806 09:32:48.058819 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":278,"threads_started":1}
I20260806 09:32:48.061374 12329 heartbeater.cc:347] Connected to a master server at 127.20.113.62:34891
I20260806 09:32:48.061689 12329 heartbeater.cc:464] Registering TS with master...
I20260806 09:32:48.062578 12329 heartbeater.cc:511] Master 127.20.113.62:34891 requested a full tablet report, sending...
I20260806 09:32:48.064241 11869 ts_manager.cc:194] Registered new tserver with Master: 71b724c2b8d64bbc9a06f26510d924f0 (127.20.113.2:32951)
I20260806 09:32:48.065063 11869 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.113.2@KRBTEST.COM'} at 127.20.113.2:58417
W20260806 09:32:48.396373 12333 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:32:48.396682 12333 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:32:48.396855 12333 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:32:48.403976 12333 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:32:48.404117 12333 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:32:48.404325 12333 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.113.3
I20260806 09:32:48.413848 12333 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:43715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.3.keytab
--principal=kudu/127.20.113.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.113.3:43133
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.113.3
--webserver_port=42761
--tserver_master_addrs=127.20.113.62:34891
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.113.3
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:32:48.415072 12333 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:32:48.416450 12333 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:32:48.430035 12344 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:32:48.430079 12341 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:32:48.430446 12342 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:32:48.430786 12333 server_base.cc:1034] running on GCE node
I20260806 09:32:48.431407 12333 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:32:48.432382 12333 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:32:48.433650 12333 hybrid_clock.cc:648] HybridClock initialized: now 1786008768433604 us; error 41 us; skew 500 ppm
I20260806 09:32:48.433966 12333 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:32:48 dist-test-slave-45dx krb5kdc[11013](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008768, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:32:48.437100 12333 init.cc:377] Logged in from keytab as kudu/127.20.113.3@KRBTEST.COM (short username kudu)
I20260806 09:32:48.438495 12333 webserver.cc:460] Webserver started at http://127.20.113.3:42761/ using document root <none> and password file <none>
I20260806 09:32:48.439165 12333 fs_manager.cc:359] Metadata directory not provided
I20260806 09:32:48.439338 12333 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:32:48.444892 12333 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260806 09:32:48.447476 12359 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:32:48.448832 12333 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 09:32:48.449020 12333 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-2/wal
uuid: "4627ec4e33c14d7987cd73370bd2aad7"
format_stamp: "Formatted at 2026-08-06 09:32:25 on dist-test-slave-45dx"
server_key: "43304b5340218053fb813ac4183e8ed2"
server_key_iv: "0505e48553258cad4ce1b9dc362b8cdd"
server_key_version: "encryptionkey@0"
I20260806 09:32:48.450030 12333 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:32:48.493387 12333 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:32:48.497431 12333 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:32:48.497671 12333 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:32:48.498886 12333 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:32:48.500263 12333 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:32:48.500368 12333 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:32:48.500447 12333 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:32:48.500501 12333 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:32:48.534590 12333 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.3:43133
I20260806 09:32:48.534677 12535 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.3:43133 every 8 connection(s)
I20260806 09:32:48.536041 12333 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-2/data/info.pb
I20260806 09:32:48.541719 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 12333
Aug 06 09:32:48 dist-test-slave-45dx krb5kdc[11013](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008768, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.3@KRBTEST.COM for kudu/127.20.113.62@KRBTEST.COM
I20260806 09:32:48.559443 12540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:48.542732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36990 (local address 127.20.113.62:34891)
0806 09:32:48.543222 (+ 490us) server_negotiation.cc:206] Beginning negotiation
0806 09:32:48.543229 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:32:48.543501 (+ 272us) server_negotiation.cc:407] Connection header received
0806 09:32:48.543712 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:32:48.543716 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:32:48.543784 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:32:48.543901 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:32:48.545642 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:48.546383 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:48.548103 (+ 1720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:48.548631 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:48.548718 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:32:48.552063 (+ 3345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:32:48.552068 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:32:48.552071 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:32:48.552100 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:32:48.556238 (+ 4138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:32:48.557061 (+ 823us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:32:48.557064 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:32:48.557066 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:32:48.557235 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:32:48.557745 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:32:48.557753 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:32:48.557755 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:32:48.558129 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:32:48.558400 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0806 09:32:48.558760 (+ 360us) server_negotiation.cc:299] Negotiation successful
0806 09:32:48.558995 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":387,"thread_start_us":276,"threads_started":1}
I20260806 09:32:48.559876 12136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:48.538701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.3:53013 (local address 127.20.113.62:34891)
0806 09:32:48.538892 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0806 09:32:48.538896 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 09:32:48.540086 (+ 1190us) server_negotiation.cc:407] Connection header received
0806 09:32:48.542029 (+ 1943us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:32:48.542037 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:32:48.542127 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:32:48.542249 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:32:48.544882 (+ 2633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:48.545603 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:48.547579 (+ 1976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:48.548011 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:48.548175 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:32:48.552367 (+ 4192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:32:48.552373 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:32:48.552377 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:32:48.552430 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:32:48.556335 (+ 3905us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:32:48.557200 (+ 865us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:32:48.557204 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:32:48.557207 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:32:48.557331 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:32:48.557884 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:32:48.557889 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:32:48.557891 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:32:48.558154 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:32:48.558302 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0806 09:32:48.559126 (+ 824us) server_negotiation.cc:299] Negotiation successful
0806 09:32:48.559513 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":45,"server-negotiator.queue_time_us":69}
I20260806 09:32:48.562230 12538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:48.539145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:34891 (local address 127.20.113.3:53013)
0806 09:32:48.539845 (+ 700us) negotiation.cc:107] Waiting for socket to connect
0806 09:32:48.539958 (+ 113us) client_negotiation.cc:174] Beginning negotiation
0806 09:32:48.541694 (+ 1736us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:32:48.543326 (+ 1632us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:32:48.543334 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:32:48.544085 (+ 751us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:32:48.544707 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:32:48.544724 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:32:48.545789 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:32:48.545795 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:32:48.547431 (+ 1636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:32:48.547447 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:32:48.548256 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:32:48.548261 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:32:48.548518 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:32:48.549259 (+ 741us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:32:48.549297 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:32:48.552160 (+ 2863us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:32:48.556472 (+ 4312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:32:48.556483 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:32:48.556498 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:32:48.557076 (+ 578us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:32:48.557456 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:32:48.557463 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:32:48.557469 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:32:48.557727 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:32:48.558277 (+ 550us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:32:48.558284 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:32:48.558701 (+ 417us) client_negotiation.cc:769] Sending connection context
0806 09:32:48.559017 (+ 316us) client_negotiation.cc:240] Negotiation successful
0806 09:32:48.560861 (+ 1844us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":283,"threads_started":1}
I20260806 09:32:48.563668 12536 heartbeater.cc:347] Connected to a master server at 127.20.113.62:34891
I20260806 09:32:48.563992 12536 heartbeater.cc:464] Registering TS with master...
I20260806 09:32:48.564718 12536 heartbeater.cc:511] Master 127.20.113.62:34891 requested a full tablet report, sending...
I20260806 09:32:48.566039 11869 ts_manager.cc:194] Registered new tserver with Master: 4627ec4e33c14d7987cd73370bd2aad7 (127.20.113.3:43133)
I20260806 09:32:48.566807 11869 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.113.3@KRBTEST.COM'} at 127.20.113.3:53013
I20260806 09:32:48.574440 20932 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 09:32:48.580798 12136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:48.575436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36998 (local address 127.20.113.62:34891)
0806 09:32:48.575633 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0806 09:32:48.575642 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 09:32:48.576009 (+ 367us) server_negotiation.cc:407] Connection header received
0806 09:32:48.576174 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:32:48.576177 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:32:48.576287 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:32:48.576453 (+ 166us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 09:32:48.576867 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:48.577592 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:48.578990 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:48.579422 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:48.579548 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:32:48.580029 (+ 481us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 09:32:48.580198 (+ 169us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0806 09:32:48.580551 (+ 353us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:36998: token expired
Metrics: {"server-negotiator.queue_time_us":60}
W20260806 09:32:48.580928 12136 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:36998: token expired
W20260806 09:32:48.581485 11753 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.20.113.62:34891)
I20260806 09:32:48.594089 12136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:48.582023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37008 (local address 127.20.113.62:34891)
0806 09:32:48.582167 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0806 09:32:48.582171 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 09:32:48.582185 (+ 14us) server_negotiation.cc:407] Connection header received
0806 09:32:48.582352 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:32:48.582356 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:32:48.582426 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:32:48.582518 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:32:48.583989 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:48.584690 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:48.586067 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:48.586506 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:48.586597 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:32:48.589199 (+ 2602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:32:48.589203 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:32:48.589206 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:32:48.589233 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:32:48.591709 (+ 2476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:32:48.592382 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:32:48.592387 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:32:48.592389 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:32:48.592505 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:32:48.592836 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:32:48.592839 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:32:48.592841 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:32:48.593135 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:32:48.593315 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0806 09:32:48.593598 (+ 283us) server_negotiation.cc:299] Negotiation successful
0806 09:32:48.593763 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 09:32:48.600432 12133 heartbeater.cc:503] Master 127.20.113.62:34891 was elected leader, sending a full tablet report...
I20260806 09:32:48.597254 11869 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37008:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 09:32:48.605206 11869 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 09:32:48.628364 12558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:48.617893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48868 (local address 127.20.113.3:43133)
0806 09:32:48.618729 (+ 836us) server_negotiation.cc:206] Beginning negotiation
0806 09:32:48.618737 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 09:32:48.619441 (+ 704us) server_negotiation.cc:407] Connection header received
0806 09:32:48.619756 (+ 315us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:32:48.619767 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:32:48.620028 (+ 261us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:32:48.621126 (+ 1098us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:32:48.621426 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:48.623154 (+ 1728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:48.625870 (+ 2716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:48.627023 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:48.627150 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:32:48.627367 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0806 09:32:48.627772 (+ 405us) server_negotiation.cc:299] Negotiation successful
0806 09:32:48.627948 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":645,"thread_start_us":464,"threads_started":1}
I20260806 09:32:48.629349 12554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:48.617111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.3:43133 (local address 127.0.0.1:48868)
0806 09:32:48.619285 (+ 2174us) negotiation.cc:107] Waiting for socket to connect
0806 09:32:48.619323 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 09:32:48.619520 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:32:48.620275 (+ 755us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:32:48.620283 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:32:48.620361 (+ 78us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:32:48.620895 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:32:48.620918 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:32:48.623318 (+ 2400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:32:48.623324 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:32:48.625719 (+ 2395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:32:48.625729 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:32:48.627248 (+ 1519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:32:48.627256 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:32:48.627507 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:32:48.627578 (+ 71us) client_negotiation.cc:769] Sending connection context
0806 09:32:48.627712 (+ 134us) client_negotiation.cc:240] Negotiation successful
0806 09:32:48.627914 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1988,"mutex_wait_us":3,"thread_start_us":1849,"threads_started":1}
I20260806 09:32:48.629402 12557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:48.617111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.2:32951 (local address 127.0.0.1:36916)
0806 09:32:48.620160 (+ 3049us) negotiation.cc:107] Waiting for socket to connect
0806 09:32:48.620179 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 09:32:48.621067 (+ 888us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:32:48.621993 (+ 926us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:32:48.621997 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:32:48.622023 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:32:48.622316 (+ 293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:32:48.622327 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:32:48.624113 (+ 1786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:32:48.624123 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:32:48.626395 (+ 2272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:32:48.626412 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:32:48.628431 (+ 2019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:32:48.628439 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:32:48.628636 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:32:48.628673 (+ 37us) client_negotiation.cc:769] Sending connection context
0806 09:32:48.628767 (+ 94us) client_negotiation.cc:240] Negotiation successful
0806 09:32:48.628913 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2909,"mutex_wait_us":91,"thread_start_us":2685,"threads_started":1}
I20260806 09:32:48.629626 12553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:48.617111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36916 (local address 127.20.113.2:32951)
0806 09:32:48.618280 (+ 1169us) server_negotiation.cc:206] Beginning negotiation
0806 09:32:48.618292 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 09:32:48.620266 (+ 1974us) server_negotiation.cc:407] Connection header received
0806 09:32:48.621258 (+ 992us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:32:48.621266 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:32:48.621492 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:32:48.621662 (+ 170us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:32:48.622533 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:48.623960 (+ 1427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:48.626582 (+ 2622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:48.627904 (+ 1322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:48.628034 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:32:48.628228 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0806 09:32:48.629101 (+ 873us) server_negotiation.cc:299] Negotiation successful
0806 09:32:48.629275 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":988,"thread_start_us":880,"threads_started":1}
I20260806 09:32:48.631639 12559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:48.619962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.1:33093 (local address 127.0.0.1:34420)
0806 09:32:48.620884 (+ 922us) negotiation.cc:107] Waiting for socket to connect
0806 09:32:48.620901 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 09:32:48.621270 (+ 369us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:32:48.624704 (+ 3434us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:32:48.624709 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:32:48.624734 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:32:48.625256 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:32:48.625271 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:32:48.626940 (+ 1669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:32:48.626947 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:32:48.629068 (+ 2121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:32:48.629086 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:32:48.630861 (+ 1775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:32:48.630868 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:32:48.631051 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:32:48.631082 (+ 31us) client_negotiation.cc:769] Sending connection context
0806 09:32:48.631176 (+ 94us) client_negotiation.cc:240] Negotiation successful
0806 09:32:48.631300 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":764,"mutex_wait_us":49,"thread_start_us":848,"threads_started":1}
I20260806 09:32:48.632664 12560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:48.620620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34420 (local address 127.20.113.1:33093)
0806 09:32:48.622924 (+ 2304us) server_negotiation.cc:206] Beginning negotiation
0806 09:32:48.622932 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 09:32:48.622957 (+ 25us) server_negotiation.cc:407] Connection header received
0806 09:32:48.623040 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:32:48.623048 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:32:48.624189 (+ 1141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:32:48.624446 (+ 257us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:32:48.625456 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:48.626814 (+ 1358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:48.629253 (+ 2439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:48.630470 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:48.630632 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:32:48.630858 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0806 09:32:48.631243 (+ 385us) server_negotiation.cc:299] Negotiation successful
0806 09:32:48.632380 (+ 1137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2126,"thread_start_us":1993,"threads_started":1}
I20260806 09:32:48.634087 12229 tablet_service.cc:1467] Processing CreateTablet for tablet d5e913e4f0544368af75ff958e7a3c32 (DEFAULT_TABLE table=test-table [id=2d7fe754bfe24343b78689f9b6c80437]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:32:48.633647 12431 tablet_service.cc:1467] Processing CreateTablet for tablet d5e913e4f0544368af75ff958e7a3c32 (DEFAULT_TABLE table=test-table [id=2d7fe754bfe24343b78689f9b6c80437]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:32:48.635561 12431 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5e913e4f0544368af75ff958e7a3c32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:32:48.635564 12229 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5e913e4f0544368af75ff958e7a3c32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:32:48.635694 12031 tablet_service.cc:1467] Processing CreateTablet for tablet d5e913e4f0544368af75ff958e7a3c32 (DEFAULT_TABLE table=test-table [id=2d7fe754bfe24343b78689f9b6c80437]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:32:48.636914 12031 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5e913e4f0544368af75ff958e7a3c32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:32:48.648286 12561 tablet_bootstrap.cc:492] T d5e913e4f0544368af75ff958e7a3c32 P 7a9ac4e4dc6d486eb682884bd5f5f027: Bootstrap starting.
I20260806 09:32:48.651163 12563 tablet_bootstrap.cc:492] T d5e913e4f0544368af75ff958e7a3c32 P 71b724c2b8d64bbc9a06f26510d924f0: Bootstrap starting.
I20260806 09:32:48.651961 12561 tablet_bootstrap.cc:654] T d5e913e4f0544368af75ff958e7a3c32 P 7a9ac4e4dc6d486eb682884bd5f5f027: Neither blocks nor log segments found. Creating new log.
I20260806 09:32:48.653437 12561 log.cc:824] T d5e913e4f0544368af75ff958e7a3c32 P 7a9ac4e4dc6d486eb682884bd5f5f027: Log is configured to *not* fsync() on all Append() calls
I20260806 09:32:48.653506 12565 tablet_bootstrap.cc:492] T d5e913e4f0544368af75ff958e7a3c32 P 4627ec4e33c14d7987cd73370bd2aad7: Bootstrap starting.
I20260806 09:32:48.655371 12563 tablet_bootstrap.cc:654] T d5e913e4f0544368af75ff958e7a3c32 P 71b724c2b8d64bbc9a06f26510d924f0: Neither blocks nor log segments found. Creating new log.
I20260806 09:32:48.656791 12563 log.cc:824] T d5e913e4f0544368af75ff958e7a3c32 P 71b724c2b8d64bbc9a06f26510d924f0: Log is configured to *not* fsync() on all Append() calls
I20260806 09:32:48.657204 12565 tablet_bootstrap.cc:654] T d5e913e4f0544368af75ff958e7a3c32 P 4627ec4e33c14d7987cd73370bd2aad7: Neither blocks nor log segments found. Creating new log.
I20260806 09:32:48.657666 12561 tablet_bootstrap.cc:492] T d5e913e4f0544368af75ff958e7a3c32 P 7a9ac4e4dc6d486eb682884bd5f5f027: No bootstrap required, opened a new log
I20260806 09:32:48.658001 12561 ts_tablet_manager.cc:1397] T d5e913e4f0544368af75ff958e7a3c32 P 7a9ac4e4dc6d486eb682884bd5f5f027: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.002s
I20260806 09:32:48.658301 12565 log.cc:824] T d5e913e4f0544368af75ff958e7a3c32 P 4627ec4e33c14d7987cd73370bd2aad7: Log is configured to *not* fsync() on all Append() calls
I20260806 09:32:48.660953 12565 tablet_bootstrap.cc:492] T d5e913e4f0544368af75ff958e7a3c32 P 4627ec4e33c14d7987cd73370bd2aad7: No bootstrap required, opened a new log
I20260806 09:32:48.661233 12565 ts_tablet_manager.cc:1397] T d5e913e4f0544368af75ff958e7a3c32 P 4627ec4e33c14d7987cd73370bd2aad7: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260806 09:32:48.662938 12563 tablet_bootstrap.cc:492] T d5e913e4f0544368af75ff958e7a3c32 P 71b724c2b8d64bbc9a06f26510d924f0: No bootstrap required, opened a new log
I20260806 09:32:48.663192 12563 ts_tablet_manager.cc:1397] T d5e913e4f0544368af75ff958e7a3c32 P 71b724c2b8d64bbc9a06f26510d924f0: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.006s
I20260806 09:32:48.668833 12565 raft_consensus.cc:348] T d5e913e4f0544368af75ff958e7a3c32 P 4627ec4e33c14d7987cd73370bd2aad7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71b724c2b8d64bbc9a06f26510d924f0" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 32951 } } peers { permanent_uuid: "4627ec4e33c14d7987cd73370bd2aad7" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 43133 } } peers { permanent_uuid: "7a9ac4e4dc6d486eb682884bd5f5f027" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33093 } }
I20260806 09:32:48.669193 12565 raft_consensus.cc:374] T d5e913e4f0544368af75ff958e7a3c32 P 4627ec4e33c14d7987cd73370bd2aad7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:32:48.668985 12561 raft_consensus.cc:348] T d5e913e4f0544368af75ff958e7a3c32 P 7a9ac4e4dc6d486eb682884bd5f5f027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71b724c2b8d64bbc9a06f26510d924f0" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 32951 } } peers { permanent_uuid: "4627ec4e33c14d7987cd73370bd2aad7" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 43133 } } peers { permanent_uuid: "7a9ac4e4dc6d486eb682884bd5f5f027" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33093 } }
I20260806 09:32:48.669303 12565 raft_consensus.cc:729] T d5e913e4f0544368af75ff958e7a3c32 P 4627ec4e33c14d7987cd73370bd2aad7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4627ec4e33c14d7987cd73370bd2aad7, State: Initialized, Role: FOLLOWER
I20260806 09:32:48.669430 12561 raft_consensus.cc:374] T d5e913e4f0544368af75ff958e7a3c32 P 7a9ac4e4dc6d486eb682884bd5f5f027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:32:48.669562 12561 raft_consensus.cc:729] T d5e913e4f0544368af75ff958e7a3c32 P 7a9ac4e4dc6d486eb682884bd5f5f027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a9ac4e4dc6d486eb682884bd5f5f027, State: Initialized, Role: FOLLOWER
I20260806 09:32:48.670027 12565 consensus_queue.cc:260] T d5e913e4f0544368af75ff958e7a3c32 P 4627ec4e33c14d7987cd73370bd2aad7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71b724c2b8d64bbc9a06f26510d924f0" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 32951 } } peers { permanent_uuid: "4627ec4e33c14d7987cd73370bd2aad7" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 43133 } } peers { permanent_uuid: "7a9ac4e4dc6d486eb682884bd5f5f027" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33093 } }
I20260806 09:32:48.670171 12561 consensus_queue.cc:260] T d5e913e4f0544368af75ff958e7a3c32 P 7a9ac4e4dc6d486eb682884bd5f5f027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71b724c2b8d64bbc9a06f26510d924f0" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 32951 } } peers { permanent_uuid: "4627ec4e33c14d7987cd73370bd2aad7" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 43133 } } peers { permanent_uuid: "7a9ac4e4dc6d486eb682884bd5f5f027" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33093 } }
I20260806 09:32:48.671259 12536 heartbeater.cc:503] Master 127.20.113.62:34891 was elected leader, sending a full tablet report...
I20260806 09:32:48.671792 12565 ts_tablet_manager.cc:1428] T d5e913e4f0544368af75ff958e7a3c32 P 4627ec4e33c14d7987cd73370bd2aad7: Time spent starting tablet: real 0.010s user 0.010s sys 0.000s
I20260806 09:32:48.675113 12563 raft_consensus.cc:348] T d5e913e4f0544368af75ff958e7a3c32 P 71b724c2b8d64bbc9a06f26510d924f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71b724c2b8d64bbc9a06f26510d924f0" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 32951 } } peers { permanent_uuid: "4627ec4e33c14d7987cd73370bd2aad7" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 43133 } } peers { permanent_uuid: "7a9ac4e4dc6d486eb682884bd5f5f027" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33093 } }
I20260806 09:32:48.675565 12563 raft_consensus.cc:374] T d5e913e4f0544368af75ff958e7a3c32 P 71b724c2b8d64bbc9a06f26510d924f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:32:48.675736 12563 raft_consensus.cc:729] T d5e913e4f0544368af75ff958e7a3c32 P 71b724c2b8d64bbc9a06f26510d924f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71b724c2b8d64bbc9a06f26510d924f0, State: Initialized, Role: FOLLOWER
I20260806 09:32:48.676407 12563 consensus_queue.cc:260] T d5e913e4f0544368af75ff958e7a3c32 P 71b724c2b8d64bbc9a06f26510d924f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71b724c2b8d64bbc9a06f26510d924f0" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 32951 } } peers { permanent_uuid: "4627ec4e33c14d7987cd73370bd2aad7" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 43133 } } peers { permanent_uuid: "7a9ac4e4dc6d486eb682884bd5f5f027" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33093 } }
I20260806 09:32:48.677765 12561 ts_tablet_manager.cc:1428] T d5e913e4f0544368af75ff958e7a3c32 P 7a9ac4e4dc6d486eb682884bd5f5f027: Time spent starting tablet: real 0.020s user 0.018s sys 0.002s
I20260806 09:32:48.678831 12329 heartbeater.cc:503] Master 127.20.113.62:34891 was elected leader, sending a full tablet report...
I20260806 09:32:48.679767 12563 ts_tablet_manager.cc:1428] T d5e913e4f0544368af75ff958e7a3c32 P 71b724c2b8d64bbc9a06f26510d924f0: Time spent starting tablet: real 0.016s user 0.014s sys 0.002s
I20260806 09:32:48.768599 12569 raft_consensus.cc:482] T d5e913e4f0544368af75ff958e7a3c32 P 4627ec4e33c14d7987cd73370bd2aad7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 09:32:48.768934 12569 raft_consensus.cc:504] T d5e913e4f0544368af75ff958e7a3c32 P 4627ec4e33c14d7987cd73370bd2aad7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71b724c2b8d64bbc9a06f26510d924f0" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 32951 } } peers { permanent_uuid: "4627ec4e33c14d7987cd73370bd2aad7" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 43133 } } peers { permanent_uuid: "7a9ac4e4dc6d486eb682884bd5f5f027" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33093 } }
I20260806 09:32:48.770875 12569 leader_election.cc:288] T d5e913e4f0544368af75ff958e7a3c32 P 4627ec4e33c14d7987cd73370bd2aad7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 71b724c2b8d64bbc9a06f26510d924f0 (127.20.113.2:32951), 7a9ac4e4dc6d486eb682884bd5f5f027 (127.20.113.1:33093)
I20260806 09:32:48.776827 12553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:48.771326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.3:44823 (local address 127.20.113.2:32951)
0806 09:32:48.771598 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0806 09:32:48.771608 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 09:32:48.771682 (+ 74us) server_negotiation.cc:407] Connection header received
0806 09:32:48.771919 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:32:48.771923 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:32:48.771982 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:32:48.772078 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:32:48.772598 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:48.773442 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:48.775142 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:48.775895 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:48.776051 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:32:48.776123 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 09:32:48.776422 (+ 299us) server_negotiation.cc:299] Negotiation successful
0806 09:32:48.776544 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260806 09:32:48.776839 12538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:48.771256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.2:32951 (local address 127.20.113.3:44823)
0806 09:32:48.771597 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0806 09:32:48.771615 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 09:32:48.771779 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:32:48.772103 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:32:48.772108 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:32:48.772142 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:32:48.772383 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:32:48.772396 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:32:48.773590 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:32:48.773596 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:32:48.774943 (+ 1347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:32:48.774967 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:32:48.776047 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:32:48.776053 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:32:48.776241 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:32:48.776263 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 09:32:48.776394 (+ 131us) client_negotiation.cc:240] Negotiation successful
0806 09:32:48.776526 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191}
I20260806 09:32:48.777570 12574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:48.771365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.1:33093 (local address 127.20.113.3:35085)
0806 09:32:48.772206 (+ 841us) negotiation.cc:107] Waiting for socket to connect
0806 09:32:48.772220 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 09:32:48.772398 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:32:48.772771 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:32:48.772775 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:32:48.772820 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:32:48.773135 (+ 315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:32:48.773143 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:32:48.774251 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:32:48.774256 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:32:48.775610 (+ 1354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:32:48.775622 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:32:48.776851 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:32:48.776858 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:32:48.777059 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:32:48.777102 (+ 43us) client_negotiation.cc:769] Sending connection context
0806 09:32:48.777187 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 09:32:48.777306 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":714,"thread_start_us":465,"threads_started":1}
I20260806 09:32:48.777637 12560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:48.771344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.3:35085 (local address 127.20.113.1:33093)
0806 09:32:48.771617 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0806 09:32:48.771623 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:32:48.772365 (+ 742us) server_negotiation.cc:407] Connection header received
0806 09:32:48.772590 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:32:48.772594 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:32:48.772652 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:32:48.772751 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:32:48.773312 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:48.774100 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:48.775770 (+ 1670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:48.776612 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:48.776817 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:32:48.776891 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 09:32:48.777222 (+ 331us) server_negotiation.cc:299] Negotiation successful
0806 09:32:48.777366 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128}
I20260806 09:32:48.777829 12263 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5e913e4f0544368af75ff958e7a3c32" candidate_uuid: "4627ec4e33c14d7987cd73370bd2aad7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71b724c2b8d64bbc9a06f26510d924f0" is_pre_election: true
I20260806 09:32:48.778398 12263 raft_consensus.cc:2454] T d5e913e4f0544368af75ff958e7a3c32 P 71b724c2b8d64bbc9a06f26510d924f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4627ec4e33c14d7987cd73370bd2aad7 in term 0.
I20260806 09:32:48.778640 12059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5e913e4f0544368af75ff958e7a3c32" candidate_uuid: "4627ec4e33c14d7987cd73370bd2aad7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a9ac4e4dc6d486eb682884bd5f5f027" is_pre_election: true
I20260806 09:32:48.779194 12367 leader_election.cc:302] T d5e913e4f0544368af75ff958e7a3c32 P 4627ec4e33c14d7987cd73370bd2aad7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4627ec4e33c14d7987cd73370bd2aad7, 71b724c2b8d64bbc9a06f26510d924f0; no voters:
I20260806 09:32:48.779237 12059 raft_consensus.cc:2454] T d5e913e4f0544368af75ff958e7a3c32 P 7a9ac4e4dc6d486eb682884bd5f5f027 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4627ec4e33c14d7987cd73370bd2aad7 in term 0.
I20260806 09:32:48.779619 12569 raft_consensus.cc:2781] T d5e913e4f0544368af75ff958e7a3c32 P 4627ec4e33c14d7987cd73370bd2aad7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 09:32:48.779778 12569 raft_consensus.cc:482] T d5e913e4f0544368af75ff958e7a3c32 P 4627ec4e33c14d7987cd73370bd2aad7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 09:32:48.779953 12569 raft_consensus.cc:3037] T d5e913e4f0544368af75ff958e7a3c32 P 4627ec4e33c14d7987cd73370bd2aad7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:32:48.782405 12569 raft_consensus.cc:504] T d5e913e4f0544368af75ff958e7a3c32 P 4627ec4e33c14d7987cd73370bd2aad7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71b724c2b8d64bbc9a06f26510d924f0" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 32951 } } peers { permanent_uuid: "4627ec4e33c14d7987cd73370bd2aad7" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 43133 } } peers { permanent_uuid: "7a9ac4e4dc6d486eb682884bd5f5f027" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33093 } }
I20260806 09:32:48.783049 12569 leader_election.cc:288] T d5e913e4f0544368af75ff958e7a3c32 P 4627ec4e33c14d7987cd73370bd2aad7 [CANDIDATE]: Term 1 election: Requested vote from peers 71b724c2b8d64bbc9a06f26510d924f0 (127.20.113.2:32951), 7a9ac4e4dc6d486eb682884bd5f5f027 (127.20.113.1:33093)
I20260806 09:32:48.783569 12059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5e913e4f0544368af75ff958e7a3c32" candidate_uuid: "4627ec4e33c14d7987cd73370bd2aad7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a9ac4e4dc6d486eb682884bd5f5f027"
I20260806 09:32:48.783586 12263 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5e913e4f0544368af75ff958e7a3c32" candidate_uuid: "4627ec4e33c14d7987cd73370bd2aad7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71b724c2b8d64bbc9a06f26510d924f0"
I20260806 09:32:48.783800 12059 raft_consensus.cc:3037] T d5e913e4f0544368af75ff958e7a3c32 P 7a9ac4e4dc6d486eb682884bd5f5f027 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:32:48.783800 12263 raft_consensus.cc:3037] T d5e913e4f0544368af75ff958e7a3c32 P 71b724c2b8d64bbc9a06f26510d924f0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:32:48.786068 12059 raft_consensus.cc:2454] T d5e913e4f0544368af75ff958e7a3c32 P 7a9ac4e4dc6d486eb682884bd5f5f027 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4627ec4e33c14d7987cd73370bd2aad7 in term 1.
I20260806 09:32:48.786131 12263 raft_consensus.cc:2454] T d5e913e4f0544368af75ff958e7a3c32 P 71b724c2b8d64bbc9a06f26510d924f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4627ec4e33c14d7987cd73370bd2aad7 in term 1.
I20260806 09:32:48.786499 12362 leader_election.cc:302] T d5e913e4f0544368af75ff958e7a3c32 P 4627ec4e33c14d7987cd73370bd2aad7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4627ec4e33c14d7987cd73370bd2aad7, 7a9ac4e4dc6d486eb682884bd5f5f027; no voters:
I20260806 09:32:48.786796 12569 raft_consensus.cc:2781] T d5e913e4f0544368af75ff958e7a3c32 P 4627ec4e33c14d7987cd73370bd2aad7 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:32:48.787246 12569 raft_consensus.cc:686] T d5e913e4f0544368af75ff958e7a3c32 P 4627ec4e33c14d7987cd73370bd2aad7 [term 1 LEADER]: Becoming Leader. State: Replica: 4627ec4e33c14d7987cd73370bd2aad7, State: Running, Role: LEADER
W20260806 09:32:48.787725 12537 tablet.cc:2369] T d5e913e4f0544368af75ff958e7a3c32 P 4627ec4e33c14d7987cd73370bd2aad7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 09:32:48.787906 12569 consensus_queue.cc:237] T d5e913e4f0544368af75ff958e7a3c32 P 4627ec4e33c14d7987cd73370bd2aad7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71b724c2b8d64bbc9a06f26510d924f0" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 32951 } } peers { permanent_uuid: "4627ec4e33c14d7987cd73370bd2aad7" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 43133 } } peers { permanent_uuid: "7a9ac4e4dc6d486eb682884bd5f5f027" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33093 } }
W20260806 09:32:48.790294 12331 tablet.cc:2369] T d5e913e4f0544368af75ff958e7a3c32 P 71b724c2b8d64bbc9a06f26510d924f0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 09:32:48.792938 11869 catalog_manager.cc:5495] T d5e913e4f0544368af75ff958e7a3c32 P 4627ec4e33c14d7987cd73370bd2aad7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4627ec4e33c14d7987cd73370bd2aad7 (127.20.113.3). New cstate: current_term: 1 leader_uuid: "4627ec4e33c14d7987cd73370bd2aad7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71b724c2b8d64bbc9a06f26510d924f0" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 32951 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4627ec4e33c14d7987cd73370bd2aad7" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 43133 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a9ac4e4dc6d486eb682884bd5f5f027" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33093 } health_report { overall_health: UNKNOWN } } }
W20260806 09:32:48.814021 12134 tablet.cc:2369] T d5e913e4f0544368af75ff958e7a3c32 P 7a9ac4e4dc6d486eb682884bd5f5f027: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 09:32:48.818375 12558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:48.812890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48882 (local address 127.20.113.3:43133)
0806 09:32:48.813069 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0806 09:32:48.813074 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:32:48.813213 (+ 139us) server_negotiation.cc:407] Connection header received
0806 09:32:48.813400 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:32:48.813404 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:32:48.813484 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:32:48.813607 (+ 123us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 09:32:48.813984 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:48.814998 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:48.816456 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:48.816868 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:48.816966 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:32:48.817449 (+ 483us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 09:32:48.817895 (+ 446us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 09:32:48.818014 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0806 09:32:48.818113 (+ 99us) server_negotiation.cc:299] Negotiation successful
0806 09:32:48.818177 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 09:32:48.826747 12059 raft_consensus.cc:1261] T d5e913e4f0544368af75ff958e7a3c32 P 7a9ac4e4dc6d486eb682884bd5f5f027 [term 1 FOLLOWER]: Refusing update from remote peer 4627ec4e33c14d7987cd73370bd2aad7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 09:32:48.826786 12263 raft_consensus.cc:1261] T d5e913e4f0544368af75ff958e7a3c32 P 71b724c2b8d64bbc9a06f26510d924f0 [term 1 FOLLOWER]: Refusing update from remote peer 4627ec4e33c14d7987cd73370bd2aad7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 09:32:48.827939 12576 consensus_queue.cc:1035] T d5e913e4f0544368af75ff958e7a3c32 P 4627ec4e33c14d7987cd73370bd2aad7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a9ac4e4dc6d486eb682884bd5f5f027" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 33093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 09:32:48.828329 12569 consensus_queue.cc:1035] T d5e913e4f0544368af75ff958e7a3c32 P 4627ec4e33c14d7987cd73370bd2aad7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "71b724c2b8d64bbc9a06f26510d924f0" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 32951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 09:32:48.839470 12581 mvcc.cc:205] Tried to move back new op lower bound from 7315491917104099328 to 7315491916959797248. Current Snapshot: MvccSnapshot[applied={T|T < 7315491917104099328}]
I20260806 09:32:48.844200 12586 mvcc.cc:205] Tried to move back new op lower bound from 7315491917104099328 to 7315491916959797248. Current Snapshot: MvccSnapshot[applied={T|T < 7315491917104099328}]
I20260806 09:32:48.854501 11866 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37008:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 09:32:48.854808 11866 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37008:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 09:32:48.858325 11866 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 271408bd5ef14ab2a87473e851c18c92: Sending DeleteTablet for 3 replicas of tablet d5e913e4f0544368af75ff958e7a3c32
I20260806 09:32:48.859472 12229 tablet_service.cc:1514] Processing DeleteTablet for tablet d5e913e4f0544368af75ff958e7a3c32 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:32:48 UTC) from {username='kudu', principal='kudu/127.20.113.62@KRBTEST.COM'} at 127.0.0.1:36916
I20260806 09:32:48.859726 12431 tablet_service.cc:1514] Processing DeleteTablet for tablet d5e913e4f0544368af75ff958e7a3c32 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:32:48 UTC) from {username='kudu', principal='kudu/127.20.113.62@KRBTEST.COM'} at 127.0.0.1:48868
I20260806 09:32:48.859730 12031 tablet_service.cc:1514] Processing DeleteTablet for tablet d5e913e4f0544368af75ff958e7a3c32 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:32:48 UTC) from {username='kudu', principal='kudu/127.20.113.62@KRBTEST.COM'} at 127.0.0.1:34420
I20260806 09:32:48.861114 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 11915
I20260806 09:32:48.871991 12594 tablet_replica.cc:330] T d5e913e4f0544368af75ff958e7a3c32 P 7a9ac4e4dc6d486eb682884bd5f5f027: stopping tablet replica
I20260806 09:32:48.872665 12594 raft_consensus.cc:2229] T d5e913e4f0544368af75ff958e7a3c32 P 7a9ac4e4dc6d486eb682884bd5f5f027 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 09:32:48.873309 12594 raft_consensus.cc:2258] T d5e913e4f0544368af75ff958e7a3c32 P 7a9ac4e4dc6d486eb682884bd5f5f027 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 09:32:48.875597 12594 ts_tablet_manager.cc:1905] T d5e913e4f0544368af75ff958e7a3c32 P 7a9ac4e4dc6d486eb682884bd5f5f027: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 09:32:48.883181 12594 ts_tablet_manager.cc:1918] T d5e913e4f0544368af75ff958e7a3c32 P 7a9ac4e4dc6d486eb682884bd5f5f027: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 09:32:48.883481 12594 log.cc:1196] T d5e913e4f0544368af75ff958e7a3c32 P 7a9ac4e4dc6d486eb682884bd5f5f027: Deleting WAL directory at /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-0/wal/wals/d5e913e4f0544368af75ff958e7a3c32
I20260806 09:32:48.884146 12594 ts_tablet_manager.cc:1939] T d5e913e4f0544368af75ff958e7a3c32 P 7a9ac4e4dc6d486eb682884bd5f5f027: Deleting consensus metadata
I20260806 09:32:48.884274 12595 tablet_replica.cc:330] T d5e913e4f0544368af75ff958e7a3c32 P 4627ec4e33c14d7987cd73370bd2aad7: stopping tablet replica
I20260806 09:32:48.885097 12595 raft_consensus.cc:2229] T d5e913e4f0544368af75ff958e7a3c32 P 4627ec4e33c14d7987cd73370bd2aad7 [term 1 LEADER]: Raft consensus shutting down.
I20260806 09:32:48.886029 12595 raft_consensus.cc:2258] T d5e913e4f0544368af75ff958e7a3c32 P 4627ec4e33c14d7987cd73370bd2aad7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 09:32:48.886475 11848 catalog_manager.cc:4841] TS 7a9ac4e4dc6d486eb682884bd5f5f027 (127.20.113.1:33093): tablet d5e913e4f0544368af75ff958e7a3c32 (table test-table [id=2d7fe754bfe24343b78689f9b6c80437]) successfully deleted
I20260806 09:32:48.884478 12560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:48.862329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34436 (local address 127.20.113.1:33093)
0806 09:32:48.862521 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0806 09:32:48.862530 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 09:32:48.862552 (+ 22us) server_negotiation.cc:407] Connection header received
0806 09:32:48.862622 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:32:48.862630 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:32:48.862744 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:32:48.862843 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:32:48.865124 (+ 2281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:48.866692 (+ 1568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:48.869160 (+ 2468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:48.870093 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:48.870217 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:32:48.874352 (+ 4135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:32:48.874372 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:32:48.874393 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:32:48.874447 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:32:48.879383 (+ 4936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:32:48.880583 (+ 1200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:32:48.880597 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:32:48.880603 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:32:48.880919 (+ 316us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:32:48.881391 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:32:48.881399 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:32:48.881405 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:32:48.882171 (+ 766us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:32:48.882494 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0806 09:32:48.882918 (+ 424us) server_negotiation.cc:299] Negotiation successful
0806 09:32:48.883236 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 09:32:48.886818 12593 tablet_replica.cc:330] T d5e913e4f0544368af75ff958e7a3c32 P 71b724c2b8d64bbc9a06f26510d924f0: stopping tablet replica
I20260806 09:32:48.887305 12593 raft_consensus.cc:2229] T d5e913e4f0544368af75ff958e7a3c32 P 71b724c2b8d64bbc9a06f26510d924f0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 09:32:48.887441 12128 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:32:48.887903 12593 raft_consensus.cc:2258] T d5e913e4f0544368af75ff958e7a3c32 P 71b724c2b8d64bbc9a06f26510d924f0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 09:32:48.892230 12595 ts_tablet_manager.cc:1905] T d5e913e4f0544368af75ff958e7a3c32 P 4627ec4e33c14d7987cd73370bd2aad7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 09:32:48.892817 12593 ts_tablet_manager.cc:1905] T d5e913e4f0544368af75ff958e7a3c32 P 71b724c2b8d64bbc9a06f26510d924f0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 09:32:48.900117 12593 ts_tablet_manager.cc:1918] T d5e913e4f0544368af75ff958e7a3c32 P 71b724c2b8d64bbc9a06f26510d924f0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 09:32:48.900369 12593 log.cc:1196] T d5e913e4f0544368af75ff958e7a3c32 P 71b724c2b8d64bbc9a06f26510d924f0: Deleting WAL directory at /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-1/wal/wals/d5e913e4f0544368af75ff958e7a3c32
I20260806 09:32:48.901006 12593 ts_tablet_manager.cc:1939] T d5e913e4f0544368af75ff958e7a3c32 P 71b724c2b8d64bbc9a06f26510d924f0: Deleting consensus metadata
I20260806 09:32:48.901780 12595 ts_tablet_manager.cc:1918] T d5e913e4f0544368af75ff958e7a3c32 P 4627ec4e33c14d7987cd73370bd2aad7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 09:32:48.902033 12595 log.cc:1196] T d5e913e4f0544368af75ff958e7a3c32 P 4627ec4e33c14d7987cd73370bd2aad7: Deleting WAL directory at /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-2/wal/wals/d5e913e4f0544368af75ff958e7a3c32
I20260806 09:32:48.902992 11851 catalog_manager.cc:4841] TS 71b724c2b8d64bbc9a06f26510d924f0 (127.20.113.2:32951): tablet d5e913e4f0544368af75ff958e7a3c32 (table test-table [id=2d7fe754bfe24343b78689f9b6c80437]) successfully deleted
I20260806 09:32:48.903302 12595 ts_tablet_manager.cc:1939] T d5e913e4f0544368af75ff958e7a3c32 P 4627ec4e33c14d7987cd73370bd2aad7: Deleting consensus metadata
I20260806 09:32:48.905153 11848 catalog_manager.cc:4841] TS 4627ec4e33c14d7987cd73370bd2aad7 (127.20.113.3:43133): tablet d5e913e4f0544368af75ff958e7a3c32 (table test-table [id=2d7fe754bfe24343b78689f9b6c80437]) successfully deleted
I20260806 09:32:48.987957 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 11915
I20260806 09:32:49.015349 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 12139
I20260806 09:32:49.032802 12553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:49.016296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36930 (local address 127.20.113.2:32951)
0806 09:32:49.016522 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0806 09:32:49.016529 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:32:49.016554 (+ 25us) server_negotiation.cc:407] Connection header received
0806 09:32:49.016643 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:32:49.016648 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:32:49.016730 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:32:49.016851 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:32:49.018092 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:49.019314 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:49.021190 (+ 1876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:49.021877 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:49.022012 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:32:49.025496 (+ 3484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:32:49.025519 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:32:49.025539 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:32:49.025582 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:32:49.029517 (+ 3935us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:32:49.030385 (+ 868us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:32:49.030396 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:32:49.030399 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:32:49.030550 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:32:49.030967 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:32:49.030972 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:32:49.030976 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:32:49.031484 (+ 508us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:32:49.031804 (+ 320us) server_negotiation.cc:1035] Waiting for connection context
0806 09:32:49.032166 (+ 362us) server_negotiation.cc:299] Negotiation successful
0806 09:32:49.032429 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 09:32:49.033595 12323 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:32:49.144975 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 12139
I20260806 09:32:49.171402 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 12333
I20260806 09:32:49.189356 12558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:32:49.172233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48892 (local address 127.20.113.3:43133)
0806 09:32:49.172495 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0806 09:32:49.172504 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 09:32:49.172672 (+ 168us) server_negotiation.cc:407] Connection header received
0806 09:32:49.172909 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:32:49.172913 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:32:49.172980 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:32:49.173074 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:32:49.175105 (+ 2031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:49.175853 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:49.177949 (+ 2096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:32:49.178386 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:32:49.178490 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:32:49.182327 (+ 3837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:32:49.182343 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:32:49.182367 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:32:49.182422 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:32:49.186127 (+ 3705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:32:49.187034 (+ 907us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:32:49.187045 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:32:49.187047 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:32:49.187206 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:32:49.187670 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:32:49.187673 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:32:49.187675 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:32:49.188031 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:32:49.188276 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0806 09:32:49.188732 (+ 456us) server_negotiation.cc:299] Negotiation successful
0806 09:32:49.188945 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260806 09:32:49.190058 12529 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:32:49.316349 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 12333
I20260806 09:32:49.344244 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 11824
I20260806 09:32:49.345638 11905 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:32:49.459036 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 11824
I20260806 09:33:09.495810 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.113.62:34891
--webserver_interface=127.20.113.62
--webserver_port=38191
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab
--principal=kudu/127.20.113.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:43715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.113.62:34891
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:33:09.840121 12603 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:33:09.840484 12603 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:33:09.840647 12603 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:33:09.847720 12603 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 09:33:09.847867 12603 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 09:33:09.847960 12603 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 09:33:09.848069 12603 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 09:33:09.848167 12603 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:33:09.848291 12603 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:33:09.848400 12603 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 09:33:09.848482 12603 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 09:33:09.858112 12603 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:43715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.20.113.62:34891
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab
--principal=kudu/127.20.113.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.113.62:34891
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.113.62
--webserver_port=38191
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:33:09.859479 12603 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:33:09.860903 12603 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:33:09.873560 12611 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:33:09.873489 12613 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:33:09.873593 12610 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:33:09.874025 12603 server_base.cc:1034] running on GCE node
I20260806 09:33:09.875149 12603 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:33:09.876672 12603 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:33:09.877939 12603 hybrid_clock.cc:648] HybridClock initialized: now 1786008789877818 us; error 62 us; skew 500 ppm
I20260806 09:33:09.878445 12603 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:33:09 dist-test-slave-45dx krb5kdc[11013](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008789, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:33:09.886902 12603 init.cc:377] Logged in from keytab as kudu/127.20.113.62@KRBTEST.COM (short username kudu)
I20260806 09:33:09.888622 12603 webserver.cc:460] Webserver started at http://127.20.113.62:38191/ using document root <none> and password file <none>
I20260806 09:33:09.889381 12603 fs_manager.cc:359] Metadata directory not provided
I20260806 09:33:09.889503 12603 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:33:09.895074 12603 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.002s
I20260806 09:33:09.898020 12622 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:33:09.899358 12603 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 09:33:09.899528 12603 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/master-0/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/master-0/wal
uuid: "271408bd5ef14ab2a87473e851c18c92"
format_stamp: "Formatted at 2026-08-06 09:32:23 on dist-test-slave-45dx"
server_key: "999c3c72152d683e38f5ee2aff82de7e"
server_key_iv: "77dc09adab2edcd0fc586ffc80d5fa28"
server_key_version: "encryptionkey@0"
I20260806 09:33:09.900554 12603 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:33:09.927933 12603 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:33:09.933535 12603 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:33:09.933859 12603 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:33:09.952134 12603 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.62:34891
I20260806 09:33:09.952167 12694 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.62:34891 every 8 connection(s)
I20260806 09:33:09.953747 12603 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
I20260806 09:33:09.960474 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 12603
I20260806 09:33:09.961024 12697 sys_catalog.cc:263] Verifying existing consensus state
I20260806 09:33:09.961369 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.113.1:33093
--local_ip_for_outbound_sockets=127.20.113.1
--tserver_master_addrs=127.20.113.62:34891
--webserver_port=42921
--webserver_interface=127.20.113.1
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.1.keytab
--principal=kudu/127.20.113.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:43715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 09:33:09.964468 12697 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 271408bd5ef14ab2a87473e851c18c92: Bootstrap starting.
I20260806 09:33:09.987166 12697 log.cc:824] T 00000000000000000000000000000000 P 271408bd5ef14ab2a87473e851c18c92: Log is configured to *not* fsync() on all Append() calls
I20260806 09:33:10.010751 12697 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 271408bd5ef14ab2a87473e851c18c92: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 09:33:10.011538 12697 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 271408bd5ef14ab2a87473e851c18c92: Bootstrap complete.
I20260806 09:33:10.025233 12697 raft_consensus.cc:348] T 00000000000000000000000000000000 P 271408bd5ef14ab2a87473e851c18c92 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "271408bd5ef14ab2a87473e851c18c92" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 34891 } }
I20260806 09:33:10.026482 12697 raft_consensus.cc:729] T 00000000000000000000000000000000 P 271408bd5ef14ab2a87473e851c18c92 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 271408bd5ef14ab2a87473e851c18c92, State: Initialized, Role: FOLLOWER
I20260806 09:33:10.027226 12697 consensus_queue.cc:260] T 00000000000000000000000000000000 P 271408bd5ef14ab2a87473e851c18c92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "271408bd5ef14ab2a87473e851c18c92" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 34891 } }
I20260806 09:33:10.027525 12697 raft_consensus.cc:388] T 00000000000000000000000000000000 P 271408bd5ef14ab2a87473e851c18c92 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 09:33:10.027673 12697 raft_consensus.cc:482] T 00000000000000000000000000000000 P 271408bd5ef14ab2a87473e851c18c92 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 09:33:10.027841 12697 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 271408bd5ef14ab2a87473e851c18c92 [term 2 FOLLOWER]: Advancing to term 3
I20260806 09:33:10.033401 12697 raft_consensus.cc:504] T 00000000000000000000000000000000 P 271408bd5ef14ab2a87473e851c18c92 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "271408bd5ef14ab2a87473e851c18c92" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 34891 } }
I20260806 09:33:10.033993 12697 leader_election.cc:302] T 00000000000000000000000000000000 P 271408bd5ef14ab2a87473e851c18c92 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 271408bd5ef14ab2a87473e851c18c92; no voters:
I20260806 09:33:10.034709 12697 leader_election.cc:288] T 00000000000000000000000000000000 P 271408bd5ef14ab2a87473e851c18c92 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260806 09:33:10.034873 12701 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 271408bd5ef14ab2a87473e851c18c92 [term 3 FOLLOWER]: Leader election won for term 3
I20260806 09:33:10.036105 12701 raft_consensus.cc:686] T 00000000000000000000000000000000 P 271408bd5ef14ab2a87473e851c18c92 [term 3 LEADER]: Becoming Leader. State: Replica: 271408bd5ef14ab2a87473e851c18c92, State: Running, Role: LEADER
I20260806 09:33:10.036705 12701 consensus_queue.cc:237] T 00000000000000000000000000000000 P 271408bd5ef14ab2a87473e851c18c92 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "271408bd5ef14ab2a87473e851c18c92" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 34891 } }
I20260806 09:33:10.036914 12697 sys_catalog.cc:564] T 00000000000000000000000000000000 P 271408bd5ef14ab2a87473e851c18c92 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 09:33:10.039124 12703 sys_catalog.cc:455] T 00000000000000000000000000000000 P 271408bd5ef14ab2a87473e851c18c92 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 271408bd5ef14ab2a87473e851c18c92. Latest consensus state: current_term: 3 leader_uuid: "271408bd5ef14ab2a87473e851c18c92" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "271408bd5ef14ab2a87473e851c18c92" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 34891 } } }
I20260806 09:33:10.039459 12703 sys_catalog.cc:458] T 00000000000000000000000000000000 P 271408bd5ef14ab2a87473e851c18c92 [sys.catalog]: This master's current role is: LEADER
I20260806 09:33:10.040048 12702 sys_catalog.cc:455] T 00000000000000000000000000000000 P 271408bd5ef14ab2a87473e851c18c92 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "271408bd5ef14ab2a87473e851c18c92" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "271408bd5ef14ab2a87473e851c18c92" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 34891 } } }
I20260806 09:33:10.040409 12702 sys_catalog.cc:458] T 00000000000000000000000000000000 P 271408bd5ef14ab2a87473e851c18c92 [sys.catalog]: This master's current role is: LEADER
I20260806 09:33:10.051108 12706 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 09:33:10.059980 12706 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 09:33:10.061520 12706 catalog_manager.cc:1259] Loaded cluster ID: a75a1d7b473849b59cfacd61a09e2517
I20260806 09:33:10.061695 12706 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 09:33:10.065843 12706 catalog_manager.cc:1504] Loading token signing keys...
I20260806 09:33:10.067482 12706 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 271408bd5ef14ab2a87473e851c18c92: Loaded TSK: 0
I20260806 09:33:10.067703 12706 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 271408bd5ef14ab2a87473e851c18c92: Loaded TSK: 1
F20260806 09:33:10.068907 12706 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fd94331f9ad google::LogMessage::Fail() at ??:0
@ 0x7fd94332288a google::LogMessage::SendToLog() at ??:0
@ 0x7fd94331f4de google::LogMessage::Flush() at ??:0
@ 0x7fd943320a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fd9470837d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7fd94707273e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7fd958d521c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7fd958d51783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7fd958ddd2a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7fd958ddd0c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7fd958e275e9 std::function<>::operator()() at ??:0
@ 0x7fd958d55cdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7fd958d53c36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fd958ddb610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7fd958ddb439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fd945f6b345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fd945f6e91c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7fd945f6e799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fd945f49eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7fd9492236db start_thread at ??:0
@ 0x7fd9421b961f clone at ??:0
W20260806 09:33:10.413852 12699 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:33:10.414141 12699 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:33:10.414273 12699 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:33:10.421329 12699 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:33:10.421428 12699 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:33:10.421630 12699 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.113.1
I20260806 09:33:10.431766 12699 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:43715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.1.keytab
--principal=kudu/127.20.113.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.113.1:33093
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.113.1
--webserver_port=42921
--tserver_master_addrs=127.20.113.62:34891
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.113.1
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:33:10.433130 12699 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:33:10.434459 12699 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:33:10.447471 12764 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:33:10.447496 12770 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:33:10.447533 12767 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:33:10.448729 12699 server_base.cc:1034] running on GCE node
I20260806 09:33:10.449277 12699 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:33:10.450114 12699 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:33:10.451349 12699 hybrid_clock.cc:648] HybridClock initialized: now 1786008790451309 us; error 53 us; skew 500 ppm
I20260806 09:33:10.451680 12699 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:33:10 dist-test-slave-45dx krb5kdc[11013](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008790, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:33:10.454953 12699 init.cc:377] Logged in from keytab as kudu/127.20.113.1@KRBTEST.COM (short username kudu)
I20260806 09:33:10.456454 12699 webserver.cc:460] Webserver started at http://127.20.113.1:42921/ using document root <none> and password file <none>
I20260806 09:33:10.457116 12699 fs_manager.cc:359] Metadata directory not provided
I20260806 09:33:10.457223 12699 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:33:10.462596 12699 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.003s
I20260806 09:33:10.465263 12778 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:33:10.466568 12699 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 09:33:10.466696 12699 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-0/wal
uuid: "7a9ac4e4dc6d486eb682884bd5f5f027"
format_stamp: "Formatted at 2026-08-06 09:32:24 on dist-test-slave-45dx"
server_key: "3ca9f2f0d1c49526f2dbb3cc9f9b72ed"
server_key_iv: "e83f8335594e7b2813f037c2985581a7"
server_key_version: "encryptionkey@0"
I20260806 09:33:10.467621 12699 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:33:10.486785 12699 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:33:10.490536 12699 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:33:10.490765 12699 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:33:10.491654 12699 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:33:10.492990 12699 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:33:10.493079 12699 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:33:10.493170 12699 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:33:10.493233 12699 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:33:10.527671 12699 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.1:33093
I20260806 09:33:10.527765 12933 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.1:33093 every 8 connection(s)
I20260806 09:33:10.529273 12699 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-0/data/info.pb
I20260806 09:33:10.533751 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 12699
I20260806 09:33:10.534076 12937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:10.532155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:34891 (local address 127.20.113.1:56741)
0806 09:33:10.532944 (+ 789us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:10.533189 (+ 245us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:34891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":354,"threads_started":1}
I20260806 09:33:10.534513 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.113.2:32951
--local_ip_for_outbound_sockets=127.20.113.2
--tserver_master_addrs=127.20.113.62:34891
--webserver_port=34949
--webserver_interface=127.20.113.2
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.2.keytab
--principal=kudu/127.20.113.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:43715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:33:10.534858 12934 heartbeater.cc:644] Failed to heartbeat to 127.20.113.62:34891 (0 consecutive failures): Network error: Failed to ping master at 127.20.113.62:34891: Client connection negotiation failed: client connection to 127.20.113.62:34891: connect: Connection refused (error 111)
I20260806 09:33:10.535697 12937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:10.535353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:34891 (local address 127.20.113.1:50185)
0806 09:33:10.535510 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:10.535587 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:34891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260806 09:33:10.536543 12937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:10.536198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:34891 (local address 127.20.113.1:50311)
0806 09:33:10.536353 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:10.536437 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:34891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260806 09:33:10.536823 12934 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 09:33:10.877445 12940 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:33:10.877722 12940 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:33:10.877895 12940 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:33:10.885036 12940 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:33:10.885156 12940 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:33:10.885341 12940 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.113.2
I20260806 09:33:10.894372 12940 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:43715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.2.keytab
--principal=kudu/127.20.113.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.113.2:32951
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.113.2
--webserver_port=34949
--tserver_master_addrs=127.20.113.62:34891
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.113.2
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:33:10.895673 12940 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:33:10.897060 12940 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:33:10.909932 12946 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:33:10.909966 12950 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:33:10.910017 12947 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:33:10.911087 12940 server_base.cc:1034] running on GCE node
I20260806 09:33:10.911702 12940 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:33:10.912683 12940 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:33:10.913906 12940 hybrid_clock.cc:648] HybridClock initialized: now 1786008790913861 us; error 42 us; skew 500 ppm
I20260806 09:33:10.914206 12940 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:33:10 dist-test-slave-45dx krb5kdc[11013](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008790, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:33:10.917469 12940 init.cc:377] Logged in from keytab as kudu/127.20.113.2@KRBTEST.COM (short username kudu)
I20260806 09:33:10.918982 12940 webserver.cc:460] Webserver started at http://127.20.113.2:34949/ using document root <none> and password file <none>
I20260806 09:33:10.919684 12940 fs_manager.cc:359] Metadata directory not provided
I20260806 09:33:10.919833 12940 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:33:10.925233 12940 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260806 09:33:10.927850 12961 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:33:10.929150 12940 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 09:33:10.929333 12940 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-1/wal
uuid: "71b724c2b8d64bbc9a06f26510d924f0"
format_stamp: "Formatted at 2026-08-06 09:32:24 on dist-test-slave-45dx"
server_key: "e2dfc7850b6c0395f8b6b9cb468d370d"
server_key_iv: "29b07e889b2b3e7e2421576eca4a105b"
server_key_version: "encryptionkey@0"
I20260806 09:33:10.930359 12940 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:33:10.954048 12940 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:33:10.958199 12940 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:33:10.958496 12940 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:33:10.959565 12940 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:33:10.960994 12940 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:33:10.961090 12940 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:33:10.961174 12940 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:33:10.961227 12940 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:33:10.996078 12940 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.2:32951
I20260806 09:33:10.996177 13113 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.2:32951 every 8 connection(s)
I20260806 09:33:10.997541 12940 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-1/data/info.pb
I20260806 09:33:10.999665 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 12940
I20260806 09:33:11.000295 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.113.3:43133
--local_ip_for_outbound_sockets=127.20.113.3
--tserver_master_addrs=127.20.113.62:34891
--webserver_port=42761
--webserver_interface=127.20.113.3
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.3.keytab
--principal=kudu/127.20.113.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:43715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 09:33:11.002575 13116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:11.000526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:34891 (local address 127.20.113.2:37591)
0806 09:33:11.001373 (+ 847us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:11.001684 (+ 311us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:34891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":553,"thread_start_us":345,"threads_started":1}
W20260806 09:33:11.003340 13114 heartbeater.cc:644] Failed to heartbeat to 127.20.113.62:34891 (0 consecutive failures): Network error: Failed to ping master at 127.20.113.62:34891: Client connection negotiation failed: client connection to 127.20.113.62:34891: connect: Connection refused (error 111)
I20260806 09:33:11.004186 13116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:11.003854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:34891 (local address 127.20.113.2:56045)
0806 09:33:11.004005 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:11.004082 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:34891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260806 09:33:11.005568 13120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:11.004647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:34891 (local address 127.20.113.2:58033)
0806 09:33:11.005224 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:11.005356 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:34891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":302,"threads_started":1}
W20260806 09:33:11.005774 13114 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 09:33:11.346925 13121 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:33:11.347208 13121 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:33:11.347407 13121 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:33:11.354971 13121 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:33:11.355134 13121 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:33:11.355335 13121 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.113.3
I20260806 09:33:11.365643 13121 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:43715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.3.keytab
--principal=kudu/127.20.113.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.113.3:43133
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.113.3
--webserver_port=42761
--tserver_master_addrs=127.20.113.62:34891
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.113.3
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:33:11.366989 13121 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:33:11.368388 13121 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:33:11.381501 13129 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:33:11.381556 13132 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:33:11.381578 13130 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:33:11.382519 13121 server_base.cc:1034] running on GCE node
I20260806 09:33:11.383224 13121 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:33:11.384119 13121 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:33:11.385357 13121 hybrid_clock.cc:648] HybridClock initialized: now 1786008791385322 us; error 47 us; skew 500 ppm
I20260806 09:33:11.385694 13121 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:33:11 dist-test-slave-45dx krb5kdc[11013](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008791, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:33:11.388814 13121 init.cc:377] Logged in from keytab as kudu/127.20.113.3@KRBTEST.COM (short username kudu)
I20260806 09:33:11.390313 13121 webserver.cc:460] Webserver started at http://127.20.113.3:42761/ using document root <none> and password file <none>
I20260806 09:33:11.391007 13121 fs_manager.cc:359] Metadata directory not provided
I20260806 09:33:11.391160 13121 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:33:11.396544 13121 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260806 09:33:11.399207 13141 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:33:11.400545 13121 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 09:33:11.400753 13121 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-2/wal
uuid: "4627ec4e33c14d7987cd73370bd2aad7"
format_stamp: "Formatted at 2026-08-06 09:32:25 on dist-test-slave-45dx"
server_key: "43304b5340218053fb813ac4183e8ed2"
server_key_iv: "0505e48553258cad4ce1b9dc362b8cdd"
server_key_version: "encryptionkey@0"
I20260806 09:33:11.401779 13121 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:33:11.427467 13121 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:33:11.432202 13121 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:33:11.432547 13121 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:33:11.433632 13121 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:33:11.435072 13121 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:33:11.435206 13121 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:33:11.435333 13121 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:33:11.435423 13121 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:33:11.470803 13121 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.3:43133
I20260806 09:33:11.470837 13297 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.3:43133 every 8 connection(s)
I20260806 09:33:11.472381 13121 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786008495763054-20932-0/minicluster-data/ts-2/data/info.pb
I20260806 09:33:11.475096 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 13121
I20260806 09:33:11.477599 13303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:11.475443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:34891 (local address 127.20.113.3:34663)
0806 09:33:11.476289 (+ 846us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:11.476628 (+ 339us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:34891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":552,"thread_start_us":359,"threads_started":1}
W20260806 09:33:11.478247 13298 heartbeater.cc:644] Failed to heartbeat to 127.20.113.62:34891 (0 consecutive failures): Network error: Failed to ping master at 127.20.113.62:34891: Client connection negotiation failed: client connection to 127.20.113.62:34891: connect: Connection refused (error 111)
I20260806 09:33:11.479084 13303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:11.478723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:34891 (local address 127.20.113.3:54311)
0806 09:33:11.478878 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:11.478943 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:34891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260806 09:33:11.480124 13303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:11.479729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:34891 (local address 127.20.113.3:39695)
0806 09:33:11.479928 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:11.480007 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:34891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260806 09:33:11.480445 13298 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 09:33:11.538015 13306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:11.537289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:34891 (local address 127.20.113.1:42621)
0806 09:33:11.537703 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:11.537801 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:34891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":204,"threads_started":1}
I20260806 09:33:12.006897 13307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:12.006229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:34891 (local address 127.20.113.2:44675)
0806 09:33:12.006626 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:12.006701 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:34891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":199,"threads_started":1}
I20260806 09:33:12.481812 13309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:12.480951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:34891 (local address 127.20.113.3:60449)
0806 09:33:12.481428 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:12.481536 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:34891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":222,"threads_started":1}
I20260806 09:33:12.539486 13311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:12.538790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:34891 (local address 127.20.113.1:40015)
0806 09:33:12.539235 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:12.539327 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:34891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":226,"threads_started":1}
I20260806 09:33:13.008558 13312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:13.007764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:34891 (local address 127.20.113.2:41401)
0806 09:33:13.008250 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:13.008396 (+ 146us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:34891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":254,"threads_started":1}
I20260806 09:33:13.483392 13314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:13.482640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:34891 (local address 127.20.113.3:56857)
0806 09:33:13.483134 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:13.483216 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:34891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":260,"threads_started":1}
I20260806 09:33:13.541043 13315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:13.540381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:34891 (local address 127.20.113.1:33639)
0806 09:33:13.540792 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:13.540870 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:34891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":210,"threads_started":1}
I20260806 09:33:14.010049 13316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:14.009365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:34891 (local address 127.20.113.2:39869)
0806 09:33:14.009797 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:14.009869 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:34891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":254,"threads_started":1}
I20260806 09:33:14.485076 13317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:14.484284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:34891 (local address 127.20.113.3:55151)
0806 09:33:14.484799 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:14.484885 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:34891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":416,"thread_start_us":237,"threads_started":1}
I20260806 09:33:14.542645 13318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:14.541907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:34891 (local address 127.20.113.1:55741)
0806 09:33:14.542361 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:14.542455 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:34891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":235,"threads_started":1}
I20260806 09:33:15.011547 13319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:15.010806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:34891 (local address 127.20.113.2:37471)
0806 09:33:15.011254 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:15.011359 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:34891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":222,"threads_started":1}
I20260806 09:33:15.486681 13320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:15.485977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:34891 (local address 127.20.113.3:47607)
0806 09:33:15.486416 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:15.486492 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:34891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":231,"threads_started":1}
I20260806 09:33:15.544263 13321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:15.543541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:34891 (local address 127.20.113.1:41319)
0806 09:33:15.544018 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:15.544098 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:34891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":260,"threads_started":1}
I20260806 09:33:16.013370 13322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:16.012661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:34891 (local address 127.20.113.2:41143)
0806 09:33:16.013107 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:16.013186 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:34891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":219,"threads_started":1}
I20260806 09:33:16.488456 13323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:16.487639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:34891 (local address 127.20.113.3:40043)
0806 09:33:16.488168 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:16.488250 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:34891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":297,"threads_started":1}
I20260806 09:33:16.545899 13325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:16.545186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:34891 (local address 127.20.113.1:42685)
0806 09:33:16.545628 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:16.545717 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:34891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":230,"threads_started":1}
I20260806 09:33:17.014922 13326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:17.014217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:34891 (local address 127.20.113.2:36819)
0806 09:33:17.014667 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:17.014746 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:34891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":224,"threads_started":1}
I20260806 09:33:17.490075 13330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:17.489351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:34891 (local address 127.20.113.3:40611)
0806 09:33:17.489816 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:17.489916 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:34891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":251,"threads_started":1}
I20260806 09:33:17.547756 13331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:17.546947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:34891 (local address 127.20.113.1:39451)
0806 09:33:17.547447 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:17.547532 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:34891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":213,"threads_started":1}
I20260806 09:33:18.016839 13332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:18.015961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:34891 (local address 127.20.113.2:52865)
0806 09:33:18.016519 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:18.016601 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:34891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":242,"threads_started":1}
I20260806 09:33:18.492954 13333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:18.491574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:34891 (local address 127.20.113.3:34975)
0806 09:33:18.492468 (+ 894us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:18.492600 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:34891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":649,"thread_start_us":366,"threads_started":1}
I20260806 09:33:18.549585 13334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:18.548910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:34891 (local address 127.20.113.1:60819)
0806 09:33:18.549348 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:18.549428 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:34891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":226,"threads_started":1}
I20260806 09:33:19.018385 13337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:19.017690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:34891 (local address 127.20.113.2:40735)
0806 09:33:19.018124 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:19.018202 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:34891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":221,"threads_started":1}
I20260806 09:33:19.494695 13339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:19.493944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:34891 (local address 127.20.113.3:55091)
0806 09:33:19.494446 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:19.494529 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:34891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":271,"threads_started":1}
I20260806 09:33:19.551374 13341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:19.550537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:34891 (local address 127.20.113.1:53293)
0806 09:33:19.551100 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:19.551181 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:34891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":316,"threads_started":1}
I20260806 09:33:20.020159 13342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:20.019399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:34891 (local address 127.20.113.2:57605)
0806 09:33:20.019887 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:20.019984 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:34891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":252,"threads_started":1}
I20260806 09:33:20.496376 13343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:20.495614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:34891 (local address 127.20.113.3:33193)
0806 09:33:20.496085 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:20.496166 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:34891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":223,"threads_started":1}
I20260806 09:33:20.553234 13345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:20.552462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:34891 (local address 127.20.113.1:48005)
0806 09:33:20.552976 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:20.553064 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:34891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":298,"threads_started":1}
I20260806 09:33:21.021786 13346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:21.021091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:34891 (local address 127.20.113.2:45003)
0806 09:33:21.021523 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:21.021604 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:34891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":256,"threads_started":1}
I20260806 09:33:21.498199 13347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:21.497415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:34891 (local address 127.20.113.3:39437)
0806 09:33:21.497945 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:21.498032 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:34891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":292,"threads_started":1}
I20260806 09:33:21.555183 13348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:21.554189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:34891 (local address 127.20.113.1:52095)
0806 09:33:21.554870 (+ 681us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:21.554994 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:34891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":342,"threads_started":1}
I20260806 09:33:22.023465 13349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:22.022762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:34891 (local address 127.20.113.2:50713)
0806 09:33:22.023212 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:22.023293 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:34891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":238,"threads_started":1}
I20260806 09:33:22.499866 13350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:22.499123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:34891 (local address 127.20.113.3:38941)
0806 09:33:22.499578 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:22.499675 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:34891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":225,"threads_started":1}
I20260806 09:33:22.556877 13351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:22.556116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:34891 (local address 127.20.113.1:34091)
0806 09:33:22.556608 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:22.556701 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:34891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":286,"threads_started":1}
I20260806 09:33:23.025254 13352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:23.024463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:34891 (local address 127.20.113.2:57097)
0806 09:33:23.024986 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:23.025077 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:34891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":223,"threads_started":1}
I20260806 09:33:23.501514 13353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:23.500832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:34891 (local address 127.20.113.3:52851)
0806 09:33:23.501265 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:23.501361 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:34891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":223,"threads_started":1}
I20260806 09:33:23.558501 13354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:23.557741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:34891 (local address 127.20.113.1:44071)
0806 09:33:23.558225 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:23.558315 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:34891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":231,"threads_started":1}
I20260806 09:33:24.026939 13355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:24.026158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:34891 (local address 127.20.113.2:35423)
0806 09:33:24.026652 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:24.026734 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:34891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":257,"threads_started":1}
I20260806 09:33:24.503271 13357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:24.502499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:34891 (local address 127.20.113.3:54171)
0806 09:33:24.502985 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:24.503080 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:34891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":225,"threads_started":1}
I20260806 09:33:24.560082 13358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:24.559363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:34891 (local address 127.20.113.1:38235)
0806 09:33:24.559799 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:24.559883 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:34891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":226,"threads_started":1}
I20260806 09:33:25.028573 13359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:25.027773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:34891 (local address 127.20.113.2:41425)
0806 09:33:25.028224 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:25.028366 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:34891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":233,"threads_started":1}
I20260806 09:33:25.504927 13360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:25.504174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:34891 (local address 127.20.113.3:43607)
0806 09:33:25.504659 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:25.504738 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:34891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":286,"threads_started":1}
I20260806 09:33:25.562003 13363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:25.561264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:34891 (local address 127.20.113.1:46253)
0806 09:33:25.561708 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:25.561816 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:34891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":241,"threads_started":1}
I20260806 09:33:26.030452 13364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:26.029670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:34891 (local address 127.20.113.2:55133)
0806 09:33:26.030144 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:26.030249 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:34891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":271,"threads_started":1}
I20260806 09:33:26.478598 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 12699
I20260806 09:33:26.496980 13365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:26.479692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40162 (local address 127.20.113.1:33093)
0806 09:33:26.480344 (+ 652us) server_negotiation.cc:206] Beginning negotiation
0806 09:33:26.480372 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0806 09:33:26.480654 (+ 282us) server_negotiation.cc:407] Connection header received
0806 09:33:26.482620 (+ 1966us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:33:26.482630 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:33:26.483049 (+ 419us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:33:26.483375 (+ 326us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:33:26.484894 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:26.486113 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:26.487361 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:26.488010 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:26.488152 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:26.490996 (+ 2844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:33:26.491003 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:33:26.491018 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:33:26.491053 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:33:26.494048 (+ 2995us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:33:26.494643 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:33:26.494649 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:33:26.494654 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:33:26.494815 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:33:26.495139 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:33:26.495142 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:33:26.495144 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:33:26.495612 (+ 468us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:33:26.495865 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0806 09:33:26.496249 (+ 384us) server_negotiation.cc:299] Negotiation successful
0806 09:33:26.496699 (+ 450us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":437,"thread_start_us":308,"threads_started":1}
I20260806 09:33:26.498426 12926 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:33:26.506771 13367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:26.506029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:34891 (local address 127.20.113.3:45923)
0806 09:33:26.506488 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:26.506596 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.62:34891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":226,"threads_started":1}
I20260806 09:33:26.581607 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 12699
I20260806 09:33:26.611101 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 12940
I20260806 09:33:26.628072 13370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:26.612095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34778 (local address 127.20.113.2:32951)
0806 09:33:26.612735 (+ 640us) server_negotiation.cc:206] Beginning negotiation
0806 09:33:26.612747 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 09:33:26.612786 (+ 39us) server_negotiation.cc:407] Connection header received
0806 09:33:26.614355 (+ 1569us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:33:26.614364 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:33:26.614778 (+ 414us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:33:26.615068 (+ 290us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:33:26.616447 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:26.617649 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:26.618952 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:26.619644 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:26.619787 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:26.622373 (+ 2586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:33:26.622381 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:33:26.622395 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:33:26.622430 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:33:26.625212 (+ 2782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:33:26.625874 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:33:26.625881 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:33:26.625887 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:33:26.626054 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:33:26.626386 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:33:26.626389 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:33:26.626391 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:33:26.626880 (+ 489us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:33:26.627122 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0806 09:33:26.627471 (+ 349us) server_negotiation.cc:299] Negotiation successful
0806 09:33:26.627781 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":458,"thread_start_us":338,"threads_started":1}
I20260806 09:33:26.629552 13107 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:33:26.711297 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 12940
I20260806 09:33:26.738396 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 13121
I20260806 09:33:26.756937 13372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:26.739498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51920 (local address 127.20.113.3:43133)
0806 09:33:26.740141 (+ 643us) server_negotiation.cc:206] Beginning negotiation
0806 09:33:26.740156 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 09:33:26.740200 (+ 44us) server_negotiation.cc:407] Connection header received
0806 09:33:26.741904 (+ 1704us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:33:26.741915 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:33:26.742341 (+ 426us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:33:26.742690 (+ 349us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:33:26.744191 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:26.745542 (+ 1351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:26.746892 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:26.747536 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:26.747748 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:26.750948 (+ 3200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:33:26.750968 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:33:26.750981 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:33:26.751016 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:33:26.753971 (+ 2955us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:33:26.754627 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:33:26.754634 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:33:26.754640 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:33:26.754806 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:33:26.755160 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:33:26.755163 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:33:26.755165 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:33:26.755661 (+ 496us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:33:26.755909 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0806 09:33:26.756240 (+ 331us) server_negotiation.cc:299] Negotiation successful
0806 09:33:26.756621 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":480,"thread_start_us":297,"threads_started":1}
I20260806 09:33:26.758528 13293 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:33:26.844173 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 13121
2026-08-06T09:33:26Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (65134 ms)
[----------] 30 tests from SecurityITest (311038 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786008495763054-20932-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 09:33:26 dist-test-slave-45dx krb5kdc[13377](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 09:33:26 dist-test-slave-45dx krb5kdc[13377](info): set up 2 sockets
Aug 06 09:33:26 dist-test-slave-45dx krb5kdc[13377](info): commencing operation
krb5kdc: starting...
W20260806 09:33:28.161336 20932 mini_kdc.cc:121] Time spent starting KDC: real 1.267s user 0.000s sys 0.056s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786008495763054-20932-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786008495763054-20932-0/krb5kdc/test-user.keytab.
Aug 06 09:33:28 dist-test-slave-45dx krb5kdc[13377](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008808, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T09:33:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T09:33:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.113.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.113.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.113.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab.
Entry for principal kudu/127.20.113.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab.
I20260806 09:33:28.496742 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786008495763054-20932-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786008495763054-20932-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786008495763054-20932-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.113.62:41393
--webserver_interface=127.20.113.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab
--principal=kudu/127.20.113.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:36595
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.113.62:41393
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:33:28.835096 13392 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:33:28.835358 13392 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:33:28.835444 13392 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:33:28.842288 13392 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 09:33:28.842387 13392 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:33:28.842450 13392 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:33:28.842507 13392 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 09:33:28.842550 13392 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 09:33:28.851727 13392 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:36595
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786008495763054-20932-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786008495763054-20932-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.113.62:41393
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab
--principal=kudu/127.20.113.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.113.62:41393
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.113.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786008495763054-20932-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:33:28.852911 13392 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:33:28.854259 13392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:33:28.866820 13399 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:33:28.866832 13400 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:33:28.866971 13405 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:33:28.867575 13392 server_base.cc:1034] running on GCE node
I20260806 09:33:28.868391 13392 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:33:28.869678 13392 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:33:28.870914 13392 hybrid_clock.cc:648] HybridClock initialized: now 1786008808870876 us; error 46 us; skew 500 ppm
I20260806 09:33:28.871246 13392 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:33:28 dist-test-slave-45dx krb5kdc[13377](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008808, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:33:28.880103 13392 init.cc:377] Logged in from keytab as kudu/127.20.113.62@KRBTEST.COM (short username kudu)
I20260806 09:33:28.881847 13392 webserver.cc:460] Webserver started at http://127.20.113.62:36401/ using document root <none> and password file <none>
I20260806 09:33:28.882591 13392 fs_manager.cc:359] Metadata directory not provided
I20260806 09:33:28.882743 13392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:33:28.883070 13392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:33:28.886274 13392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786008495763054-20932-0/minicluster-data/master-0/data/instance:
uuid: "e1ebfd4144014ce5975e107f5d969261"
format_stamp: "Formatted at 2026-08-06 09:33:28 on dist-test-slave-45dx"
server_key: "009e92739a332a3a097e558ca2577c18"
server_key_iv: "f2874a1f34f1eb0e2d03bde19a0785c2"
server_key_version: "encryptionkey@0"
I20260806 09:33:28.887089 13392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786008495763054-20932-0/minicluster-data/master-0/wal/instance:
uuid: "e1ebfd4144014ce5975e107f5d969261"
format_stamp: "Formatted at 2026-08-06 09:33:28 on dist-test-slave-45dx"
server_key: "009e92739a332a3a097e558ca2577c18"
server_key_iv: "f2874a1f34f1eb0e2d03bde19a0785c2"
server_key_version: "encryptionkey@0"
I20260806 09:33:28.892666 13392 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260806 09:33:28.896178 13417 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:33:28.897593 13392 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 09:33:28.897806 13392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786008495763054-20932-0/minicluster-data/master-0/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786008495763054-20932-0/minicluster-data/master-0/wal
uuid: "e1ebfd4144014ce5975e107f5d969261"
format_stamp: "Formatted at 2026-08-06 09:33:28 on dist-test-slave-45dx"
server_key: "009e92739a332a3a097e558ca2577c18"
server_key_iv: "f2874a1f34f1eb0e2d03bde19a0785c2"
server_key_version: "encryptionkey@0"
I20260806 09:33:28.898026 13392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786008495763054-20932-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786008495763054-20932-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786008495763054-20932-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:33:28.916611 13392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:33:28.920593 13392 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:33:28.920897 13392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:33:28.939453 13392 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.62:41393
I20260806 09:33:28.939487 13488 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.62:41393 every 8 connection(s)
I20260806 09:33:28.940805 13392 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
I20260806 09:33:28.944531 13489 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:33:28.949401 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 13392
I20260806 09:33:28.949620 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786008495763054-20932-0/minicluster-data/master-0/wal/instance
I20260806 09:33:28.950407 20932 external_mini_cluster.cc:1428] Setting key 2ab4b859b019001023547fa6887d5632
I20260806 09:33:28.955442 13489 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e1ebfd4144014ce5975e107f5d969261: Bootstrap starting.
I20260806 09:33:28.959072 13489 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e1ebfd4144014ce5975e107f5d969261: Neither blocks nor log segments found. Creating new log.
I20260806 09:33:28.960274 13489 log.cc:824] T 00000000000000000000000000000000 P e1ebfd4144014ce5975e107f5d969261: Log is configured to *not* fsync() on all Append() calls
I20260806 09:33:28.963634 13489 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e1ebfd4144014ce5975e107f5d969261: No bootstrap required, opened a new log
Aug 06 09:33:28 dist-test-slave-45dx krb5kdc[13377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008808, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.113.62@KRBTEST.COM
I20260806 09:33:28.971944 13489 raft_consensus.cc:348] T 00000000000000000000000000000000 P e1ebfd4144014ce5975e107f5d969261 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1ebfd4144014ce5975e107f5d969261" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 41393 } }
I20260806 09:33:28.972337 13489 raft_consensus.cc:374] T 00000000000000000000000000000000 P e1ebfd4144014ce5975e107f5d969261 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:33:28.972465 13489 raft_consensus.cc:729] T 00000000000000000000000000000000 P e1ebfd4144014ce5975e107f5d969261 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1ebfd4144014ce5975e107f5d969261, State: Initialized, Role: FOLLOWER
I20260806 09:33:28.973194 13489 consensus_queue.cc:260] T 00000000000000000000000000000000 P e1ebfd4144014ce5975e107f5d969261 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1ebfd4144014ce5975e107f5d969261" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 41393 } }
I20260806 09:33:28.973492 13489 raft_consensus.cc:388] T 00000000000000000000000000000000 P e1ebfd4144014ce5975e107f5d969261 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 09:33:28.973632 13489 raft_consensus.cc:482] T 00000000000000000000000000000000 P e1ebfd4144014ce5975e107f5d969261 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 09:33:28.973829 13489 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e1ebfd4144014ce5975e107f5d969261 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:33:28.974710 13492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:28.952520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49168 (local address 127.20.113.62:41393)
0806 09:33:28.953376 (+ 856us) server_negotiation.cc:206] Beginning negotiation
0806 09:33:28.953387 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 09:33:28.953430 (+ 43us) server_negotiation.cc:407] Connection header received
0806 09:33:28.955010 (+ 1580us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:33:28.955030 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:33:28.955375 (+ 345us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:33:28.955718 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:33:28.957736 (+ 2018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:28.958941 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:28.960802 (+ 1861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:28.961859 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:28.961989 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:28.966814 (+ 4825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:33:28.966821 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:33:28.966838 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:33:28.966885 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:33:28.970126 (+ 3241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:33:28.970859 (+ 733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:33:28.970866 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:33:28.970871 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:33:28.971040 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:33:28.971475 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:33:28.971479 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:33:28.971481 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:33:28.972022 (+ 541us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:33:28.972346 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0806 09:33:28.972806 (+ 460us) server_negotiation.cc:299] Negotiation successful
0806 09:33:28.973316 (+ 510us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":639,"thread_start_us":414,"threads_started":1}
I20260806 09:33:28.976941 13489 raft_consensus.cc:504] T 00000000000000000000000000000000 P e1ebfd4144014ce5975e107f5d969261 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1ebfd4144014ce5975e107f5d969261" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 41393 } }
I20260806 09:33:28.977481 13489 leader_election.cc:302] T 00000000000000000000000000000000 P e1ebfd4144014ce5975e107f5d969261 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e1ebfd4144014ce5975e107f5d969261; no voters:
I20260806 09:33:28.978128 13489 leader_election.cc:288] T 00000000000000000000000000000000 P e1ebfd4144014ce5975e107f5d969261 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 09:33:28.978248 13495 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e1ebfd4144014ce5975e107f5d969261 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:33:28.978948 13495 raft_consensus.cc:686] T 00000000000000000000000000000000 P e1ebfd4144014ce5975e107f5d969261 [term 1 LEADER]: Becoming Leader. State: Replica: e1ebfd4144014ce5975e107f5d969261, State: Running, Role: LEADER
I20260806 09:33:28.979554 13495 consensus_queue.cc:237] T 00000000000000000000000000000000 P e1ebfd4144014ce5975e107f5d969261 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1ebfd4144014ce5975e107f5d969261" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 41393 } }
I20260806 09:33:28.980458 13489 sys_catalog.cc:564] T 00000000000000000000000000000000 P e1ebfd4144014ce5975e107f5d969261 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 09:33:28.982471 13499 sys_catalog.cc:455] T 00000000000000000000000000000000 P e1ebfd4144014ce5975e107f5d969261 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e1ebfd4144014ce5975e107f5d969261. Latest consensus state: current_term: 1 leader_uuid: "e1ebfd4144014ce5975e107f5d969261" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1ebfd4144014ce5975e107f5d969261" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 41393 } } }
I20260806 09:33:28.982589 13498 sys_catalog.cc:455] T 00000000000000000000000000000000 P e1ebfd4144014ce5975e107f5d969261 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e1ebfd4144014ce5975e107f5d969261" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1ebfd4144014ce5975e107f5d969261" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 41393 } } }
I20260806 09:33:28.982905 13499 sys_catalog.cc:458] T 00000000000000000000000000000000 P e1ebfd4144014ce5975e107f5d969261 [sys.catalog]: This master's current role is: LEADER
I20260806 09:33:28.982995 13498 sys_catalog.cc:458] T 00000000000000000000000000000000 P e1ebfd4144014ce5975e107f5d969261 [sys.catalog]: This master's current role is: LEADER
I20260806 09:33:28.984741 13505 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 09:33:28.989303 13505 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 09:33:28.998018 13505 catalog_manager.cc:1347] Generated new cluster ID: 4f8c47b7f7084105b234965ab316b46a
I20260806 09:33:28.998119 13505 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 09:33:29.010069 13505 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 09:33:29.011305 13505 catalog_manager.cc:1504] Loading token signing keys...
I20260806 09:33:29.024089 13505 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e1ebfd4144014ce5975e107f5d969261: Generated new TSK 0
I20260806 09:33:29.024909 13505 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.113.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.113.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.113.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.1.keytab.
Entry for principal kudu/127.20.113.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.1.keytab.
I20260806 09:33:29.117475 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786008495763054-20932-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786008495763054-20932-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786008495763054-20932-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786008495763054-20932-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.113.1:0
--local_ip_for_outbound_sockets=127.20.113.1
--webserver_interface=127.20.113.1
--webserver_port=0
--tserver_master_addrs=127.20.113.62:41393
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.1.keytab
--principal=kudu/127.20.113.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:36595
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:33:29.473459 13525 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:33:29.473762 13525 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:33:29.473881 13525 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:33:29.481446 13525 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:33:29.481554 13525 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:33:29.481933 13525 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.113.1
I20260806 09:33:29.492399 13525 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:36595
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786008495763054-20932-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786008495763054-20932-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.1.keytab
--principal=kudu/127.20.113.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.113.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786008495763054-20932-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.113.1
--webserver_port=0
--tserver_master_addrs=127.20.113.62:41393
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.113.1
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786008495763054-20932-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:33:29.493757 13525 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:33:29.495150 13525 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:33:29.508833 13530 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:33:29.508790 13537 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:33:29.508805 13531 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:33:29.509713 13525 server_base.cc:1034] running on GCE node
I20260806 09:33:29.510284 13525 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:33:29.511152 13525 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:33:29.512390 13525 hybrid_clock.cc:648] HybridClock initialized: now 1786008809512321 us; error 56 us; skew 500 ppm
I20260806 09:33:29.512677 13525 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:33:29 dist-test-slave-45dx krb5kdc[13377](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008809, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:33:29.515795 13525 init.cc:377] Logged in from keytab as kudu/127.20.113.1@KRBTEST.COM (short username kudu)
I20260806 09:33:29.517278 13525 webserver.cc:460] Webserver started at http://127.20.113.1:43159/ using document root <none> and password file <none>
I20260806 09:33:29.517984 13525 fs_manager.cc:359] Metadata directory not provided
I20260806 09:33:29.518152 13525 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:33:29.518492 13525 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:33:29.521483 13525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786008495763054-20932-0/minicluster-data/ts-0/data/instance:
uuid: "4faa2a9033fd4777adb64589c43b9df2"
format_stamp: "Formatted at 2026-08-06 09:33:29 on dist-test-slave-45dx"
server_key: "0418b16c5015c343eacc8f8995364d8c"
server_key_iv: "e0cee216d6702d15b9229b8a1a3ddc38"
server_key_version: "encryptionkey@0"
I20260806 09:33:29.522290 13525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786008495763054-20932-0/minicluster-data/ts-0/wal/instance:
uuid: "4faa2a9033fd4777adb64589c43b9df2"
format_stamp: "Formatted at 2026-08-06 09:33:29 on dist-test-slave-45dx"
server_key: "0418b16c5015c343eacc8f8995364d8c"
server_key_iv: "e0cee216d6702d15b9229b8a1a3ddc38"
server_key_version: "encryptionkey@0"
I20260806 09:33:29.527431 13525 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260806 09:33:29.530838 13550 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:33:29.532042 13525 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260806 09:33:29.532223 13525 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786008495763054-20932-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786008495763054-20932-0/minicluster-data/ts-0/wal
uuid: "4faa2a9033fd4777adb64589c43b9df2"
format_stamp: "Formatted at 2026-08-06 09:33:29 on dist-test-slave-45dx"
server_key: "0418b16c5015c343eacc8f8995364d8c"
server_key_iv: "e0cee216d6702d15b9229b8a1a3ddc38"
server_key_version: "encryptionkey@0"
I20260806 09:33:29.532521 13525 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786008495763054-20932-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786008495763054-20932-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786008495763054-20932-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:33:29.562160 13525 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:33:29.566845 13525 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:33:29.567158 13525 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:33:29.568195 13525 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:33:29.569562 13525 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:33:29.569665 13525 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:33:29.569816 13525 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:33:29.569906 13525 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:33:29.603809 13525 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.1:39175
I20260806 09:33:29.603935 13706 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.1:39175 every 8 connection(s)
I20260806 09:33:29.605105 13525 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786008495763054-20932-0/minicluster-data/ts-0/data/info.pb
I20260806 09:33:29.614656 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 13525
I20260806 09:33:29.614879 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786008495763054-20932-0/minicluster-data/ts-0/wal/instance
I20260806 09:33:29.615741 20932 external_mini_cluster.cc:1428] Setting key 2e329b467a3fe969c0e6a5a3bf1c67a6
Aug 06 09:33:29 dist-test-slave-45dx krb5kdc[13377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008809, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.1@KRBTEST.COM for kudu/127.20.113.62@KRBTEST.COM
I20260806 09:33:29.626257 13709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:29.607624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.1:34101 (local address 127.20.113.62:41393)
0806 09:33:29.608074 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0806 09:33:29.608078 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 09:33:29.609065 (+ 987us) server_negotiation.cc:407] Connection header received
0806 09:33:29.610903 (+ 1838us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:33:29.610909 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:33:29.611021 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:33:29.611144 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:33:29.612813 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:29.613569 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:29.615854 (+ 2285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:29.616352 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:29.616523 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:29.620515 (+ 3992us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:33:29.620520 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:33:29.620523 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:33:29.620554 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:33:29.623186 (+ 2632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:33:29.623903 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:33:29.623915 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:33:29.623918 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:33:29.624031 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:33:29.624539 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:33:29.624546 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:33:29.624553 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:33:29.624802 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:33:29.624960 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0806 09:33:29.625649 (+ 689us) server_negotiation.cc:299] Negotiation successful
0806 09:33:29.625881 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":215,"threads_started":1}
I20260806 09:33:29.627245 13710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:29.608176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:41393 (local address 127.20.113.1:34101)
0806 09:33:29.608898 (+ 722us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:29.608945 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 09:33:29.610610 (+ 1665us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:33:29.611401 (+ 791us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:33:29.611410 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:33:29.611949 (+ 539us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:33:29.612631 (+ 682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:33:29.612651 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:33:29.613737 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:33:29.613741 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:33:29.615669 (+ 1928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:33:29.615686 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:33:29.616593 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:33:29.616598 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:33:29.616912 (+ 314us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:29.617687 (+ 775us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:33:29.617720 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:33:29.620251 (+ 2531us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:33:29.623340 (+ 3089us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:33:29.623348 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:33:29.623363 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:33:29.623772 (+ 409us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:33:29.624143 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:33:29.624147 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:33:29.624151 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:33:29.624424 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:33:29.624934 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:33:29.624942 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:33:29.625279 (+ 337us) client_negotiation.cc:769] Sending connection context
0806 09:33:29.625533 (+ 254us) client_negotiation.cc:240] Negotiation successful
0806 09:33:29.626059 (+ 526us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":307,"threads_started":1}
I20260806 09:33:29.628863 13707 heartbeater.cc:347] Connected to a master server at 127.20.113.62:41393
I20260806 09:33:29.629164 13707 heartbeater.cc:464] Registering TS with master...
I20260806 09:33:29.629856 13707 heartbeater.cc:511] Master 127.20.113.62:41393 requested a full tablet report, sending...
I20260806 09:33:29.632133 13446 ts_manager.cc:194] Registered new tserver with Master: 4faa2a9033fd4777adb64589c43b9df2 (127.20.113.1:39175)
I20260806 09:33:29.634222 13446 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.113.1@KRBTEST.COM'} at 127.20.113.1:34101
WARNING: no policy specified for kudu/127.20.113.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.113.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.113.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.2.keytab.
Entry for principal kudu/127.20.113.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.2.keytab.
I20260806 09:33:29.690117 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786008495763054-20932-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786008495763054-20932-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786008495763054-20932-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786008495763054-20932-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.113.2:0
--local_ip_for_outbound_sockets=127.20.113.2
--webserver_interface=127.20.113.2
--webserver_port=0
--tserver_master_addrs=127.20.113.62:41393
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.2.keytab
--principal=kudu/127.20.113.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:36595
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:33:30.039197 13713 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:33:30.039489 13713 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:33:30.039583 13713 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:33:30.046674 13713 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:33:30.046774 13713 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:33:30.046895 13713 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.113.2
I20260806 09:33:30.056296 13713 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:36595
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786008495763054-20932-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786008495763054-20932-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.2.keytab
--principal=kudu/127.20.113.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.113.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786008495763054-20932-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.113.2
--webserver_port=0
--tserver_master_addrs=127.20.113.62:41393
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.113.2
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786008495763054-20932-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:33:30.057493 13713 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:33:30.058717 13713 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:33:30.072614 13720 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:33:30.072561 13728 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:33:30.072593 13721 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:33:30.073336 13713 server_base.cc:1034] running on GCE node
I20260806 09:33:30.073869 13713 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:33:30.074717 13713 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:33:30.075906 13713 hybrid_clock.cc:648] HybridClock initialized: now 1786008810075850 us; error 52 us; skew 500 ppm
I20260806 09:33:30.076180 13713 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:33:30 dist-test-slave-45dx krb5kdc[13377](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008810, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:33:30.079465 13713 init.cc:377] Logged in from keytab as kudu/127.20.113.2@KRBTEST.COM (short username kudu)
I20260806 09:33:30.081053 13713 webserver.cc:460] Webserver started at http://127.20.113.2:45141/ using document root <none> and password file <none>
I20260806 09:33:30.081709 13713 fs_manager.cc:359] Metadata directory not provided
I20260806 09:33:30.081903 13713 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:33:30.082221 13713 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:33:30.085724 13713 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786008495763054-20932-0/minicluster-data/ts-1/data/instance:
uuid: "a696398c785a478b8810d2b956faa8b2"
format_stamp: "Formatted at 2026-08-06 09:33:30 on dist-test-slave-45dx"
server_key: "9d0fa61ca48eb1ab6cb3e675a8100d57"
server_key_iv: "8980dede5749356e3b4387af492676e9"
server_key_version: "encryptionkey@0"
I20260806 09:33:30.086504 13713 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786008495763054-20932-0/minicluster-data/ts-1/wal/instance:
uuid: "a696398c785a478b8810d2b956faa8b2"
format_stamp: "Formatted at 2026-08-06 09:33:30 on dist-test-slave-45dx"
server_key: "9d0fa61ca48eb1ab6cb3e675a8100d57"
server_key_iv: "8980dede5749356e3b4387af492676e9"
server_key_version: "encryptionkey@0"
I20260806 09:33:30.091871 13713 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260806 09:33:30.095350 13735 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:33:30.096747 13713 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260806 09:33:30.096946 13713 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786008495763054-20932-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786008495763054-20932-0/minicluster-data/ts-1/wal
uuid: "a696398c785a478b8810d2b956faa8b2"
format_stamp: "Formatted at 2026-08-06 09:33:30 on dist-test-slave-45dx"
server_key: "9d0fa61ca48eb1ab6cb3e675a8100d57"
server_key_iv: "8980dede5749356e3b4387af492676e9"
server_key_version: "encryptionkey@0"
I20260806 09:33:30.097147 13713 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786008495763054-20932-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786008495763054-20932-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786008495763054-20932-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:33:30.120093 13713 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:33:30.123912 13713 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:33:30.124221 13713 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:33:30.125207 13713 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:33:30.126513 13713 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:33:30.126631 13713 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:33:30.126788 13713 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:33:30.126864 13713 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:33:30.160997 13713 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.2:46655
I20260806 09:33:30.161082 13900 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.2:46655 every 8 connection(s)
I20260806 09:33:30.162338 13713 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786008495763054-20932-0/minicluster-data/ts-1/data/info.pb
I20260806 09:33:30.166141 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 13713
I20260806 09:33:30.166374 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786008495763054-20932-0/minicluster-data/ts-1/wal/instance
I20260806 09:33:30.167038 20932 external_mini_cluster.cc:1428] Setting key b7258c368ea49b814699cc5f823a277d
Aug 06 09:33:30 dist-test-slave-45dx krb5kdc[13377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008810, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.2@KRBTEST.COM for kudu/127.20.113.62@KRBTEST.COM
I20260806 09:33:30.185390 13906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:30.164869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.2:45511 (local address 127.20.113.62:41393)
0806 09:33:30.165352 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0806 09:33:30.165357 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:33:30.166528 (+ 1171us) server_negotiation.cc:407] Connection header received
0806 09:33:30.168614 (+ 2086us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:33:30.168620 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:33:30.168692 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:33:30.168836 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:33:30.170886 (+ 2050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:30.171575 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:30.174026 (+ 2451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:30.174462 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:30.174570 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:30.178962 (+ 4392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:33:30.178968 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:33:30.178971 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:33:30.179005 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:33:30.182084 (+ 3079us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:33:30.182872 (+ 788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:33:30.182881 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:33:30.182884 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:33:30.183014 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:33:30.183484 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:33:30.183488 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:33:30.183490 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:33:30.183766 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:33:30.183960 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0806 09:33:30.184760 (+ 800us) server_negotiation.cc:299] Negotiation successful
0806 09:33:30.185000 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":237,"threads_started":1}
I20260806 09:33:30.186419 13907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:30.165311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:41393 (local address 127.20.113.2:45511)
0806 09:33:30.166205 (+ 894us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:30.166256 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 09:33:30.168358 (+ 2102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:33:30.169104 (+ 746us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:33:30.169112 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:33:30.169894 (+ 782us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:33:30.170695 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:33:30.170722 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:33:30.171758 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:33:30.171766 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:33:30.173858 (+ 2092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:33:30.173877 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:33:30.174694 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:33:30.174703 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:33:30.175023 (+ 320us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:30.175885 (+ 862us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:33:30.175918 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:33:30.178689 (+ 2771us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:33:30.182231 (+ 3542us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:33:30.182240 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:33:30.182257 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:33:30.182732 (+ 475us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:33:30.183134 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:33:30.183139 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:33:30.183143 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:33:30.183349 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:33:30.183940 (+ 591us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:33:30.183947 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:33:30.184287 (+ 340us) client_negotiation.cc:769] Sending connection context
0806 09:33:30.184633 (+ 346us) client_negotiation.cc:240] Negotiation successful
0806 09:33:30.185201 (+ 568us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":603,"thread_start_us":330,"threads_started":1}
I20260806 09:33:30.188006 13901 heartbeater.cc:347] Connected to a master server at 127.20.113.62:41393
I20260806 09:33:30.188395 13901 heartbeater.cc:464] Registering TS with master...
I20260806 09:33:30.189277 13901 heartbeater.cc:511] Master 127.20.113.62:41393 requested a full tablet report, sending...
I20260806 09:33:30.190944 13446 ts_manager.cc:194] Registered new tserver with Master: a696398c785a478b8810d2b956faa8b2 (127.20.113.2:46655)
I20260806 09:33:30.191781 13446 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.113.2@KRBTEST.COM'} at 127.20.113.2:45511
WARNING: no policy specified for kudu/127.20.113.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.113.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.113.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.3.keytab.
Entry for principal kudu/127.20.113.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.3.keytab.
I20260806 09:33:30.247416 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786008495763054-20932-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786008495763054-20932-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786008495763054-20932-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786008495763054-20932-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.113.3:0
--local_ip_for_outbound_sockets=127.20.113.3
--webserver_interface=127.20.113.3
--webserver_port=0
--tserver_master_addrs=127.20.113.62:41393
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.3.keytab
--principal=kudu/127.20.113.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:36595
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:33:30.598316 13911 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:33:30.598579 13911 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:33:30.598685 13911 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:33:30.606056 13911 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:33:30.606150 13911 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:33:30.606354 13911 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.113.3
I20260806 09:33:30.615710 13911 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:36595
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786008495763054-20932-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786008495763054-20932-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.3.keytab
--principal=kudu/127.20.113.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.113.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786008495763054-20932-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.113.3
--webserver_port=0
--tserver_master_addrs=127.20.113.62:41393
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.113.3
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786008495763054-20932-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:33:30.616973 13911 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:33:30.618245 13911 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:33:30.630858 13917 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:33:30.630868 13919 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:33:30.630904 13921 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:33:30.631865 13911 server_base.cc:1034] running on GCE node
I20260806 09:33:30.632520 13911 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:33:30.633371 13911 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:33:30.634578 13911 hybrid_clock.cc:648] HybridClock initialized: now 1786008810634539 us; error 27 us; skew 500 ppm
I20260806 09:33:30.634859 13911 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 09:33:30.637284 13707 heartbeater.cc:503] Master 127.20.113.62:41393 was elected leader, sending a full tablet report...
Aug 06 09:33:30 dist-test-slave-45dx krb5kdc[13377](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008810, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:33:30.638533 13911 init.cc:377] Logged in from keytab as kudu/127.20.113.3@KRBTEST.COM (short username kudu)
I20260806 09:33:30.640219 13911 webserver.cc:460] Webserver started at http://127.20.113.3:33699/ using document root <none> and password file <none>
I20260806 09:33:30.641017 13911 fs_manager.cc:359] Metadata directory not provided
I20260806 09:33:30.641135 13911 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:33:30.641515 13911 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:33:30.644721 13911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786008495763054-20932-0/minicluster-data/ts-2/data/instance:
uuid: "8db53ae1e86248858c09c8aca3c43655"
format_stamp: "Formatted at 2026-08-06 09:33:30 on dist-test-slave-45dx"
server_key: "585a3a3c7d9b91d7fcbd59cc3e5495f0"
server_key_iv: "ea2044f4976a9a2fbeb72142ba66563a"
server_key_version: "encryptionkey@0"
I20260806 09:33:30.645511 13911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786008495763054-20932-0/minicluster-data/ts-2/wal/instance:
uuid: "8db53ae1e86248858c09c8aca3c43655"
format_stamp: "Formatted at 2026-08-06 09:33:30 on dist-test-slave-45dx"
server_key: "585a3a3c7d9b91d7fcbd59cc3e5495f0"
server_key_iv: "ea2044f4976a9a2fbeb72142ba66563a"
server_key_version: "encryptionkey@0"
I20260806 09:33:30.650784 13911 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 09:33:30.654166 13928 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:33:30.655458 13911 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 09:33:30.655637 13911 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786008495763054-20932-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786008495763054-20932-0/minicluster-data/ts-2/wal
uuid: "8db53ae1e86248858c09c8aca3c43655"
format_stamp: "Formatted at 2026-08-06 09:33:30 on dist-test-slave-45dx"
server_key: "585a3a3c7d9b91d7fcbd59cc3e5495f0"
server_key_iv: "ea2044f4976a9a2fbeb72142ba66563a"
server_key_version: "encryptionkey@0"
I20260806 09:33:30.655839 13911 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786008495763054-20932-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786008495763054-20932-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786008495763054-20932-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:33:30.694720 13911 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:33:30.699004 13911 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:33:30.699363 13911 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:33:30.700449 13911 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:33:30.702013 13911 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:33:30.702134 13911 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:33:30.702310 13911 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:33:30.702418 13911 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:33:30.739598 13911 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.3:41867
I20260806 09:33:30.739683 14095 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.3:41867 every 8 connection(s)
I20260806 09:33:30.741052 13911 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786008495763054-20932-0/minicluster-data/ts-2/data/info.pb
I20260806 09:33:30.743132 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 13911
I20260806 09:33:30.743368 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786008495763054-20932-0/minicluster-data/ts-2/wal/instance
I20260806 09:33:30.743852 20932 external_mini_cluster.cc:1428] Setting key 7270101657b1bbfdd69773e6147ebfda
Aug 06 09:33:30 dist-test-slave-45dx krb5kdc[13377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008810, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.3@KRBTEST.COM for kudu/127.20.113.62@KRBTEST.COM
I20260806 09:33:30.762729 14102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:30.743928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.3:38699 (local address 127.20.113.62:41393)
0806 09:33:30.744561 (+ 633us) server_negotiation.cc:206] Beginning negotiation
0806 09:33:30.744568 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:33:30.745794 (+ 1226us) server_negotiation.cc:407] Connection header received
0806 09:33:30.748202 (+ 2408us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:33:30.748207 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:33:30.748358 (+ 151us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:33:30.748488 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:33:30.750000 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:30.750782 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:30.752290 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:30.752732 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:30.752893 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:30.756442 (+ 3549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:33:30.756448 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:33:30.756451 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:33:30.756484 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:33:30.759766 (+ 3282us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:33:30.760713 (+ 947us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:33:30.760717 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:33:30.760720 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:33:30.760874 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:33:30.761297 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:33:30.761300 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:33:30.761302 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:33:30.761550 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:33:30.761665 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 09:33:30.762307 (+ 642us) server_negotiation.cc:299] Negotiation successful
0806 09:33:30.762464 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":492,"thread_start_us":278,"threads_started":1}
I20260806 09:33:30.763687 14105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:30.744623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:41393 (local address 127.20.113.3:38699)
0806 09:33:30.745627 (+ 1004us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:30.745667 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 09:33:30.747938 (+ 2271us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:33:30.748737 (+ 799us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:33:30.748745 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:33:30.749274 (+ 529us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:33:30.749822 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:33:30.749840 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:33:30.750937 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:33:30.750945 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:33:30.752138 (+ 1193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:33:30.752150 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:33:30.753007 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:33:30.753015 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:33:30.753263 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:30.753976 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:33:30.753999 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:33:30.756201 (+ 2202us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:33:30.759929 (+ 3728us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:33:30.759936 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:33:30.759947 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:33:30.760590 (+ 643us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:33:30.760970 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:33:30.760974 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:33:30.760976 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:33:30.761192 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:33:30.761656 (+ 464us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:33:30.761661 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:33:30.761967 (+ 306us) client_negotiation.cc:769] Sending connection context
0806 09:33:30.762186 (+ 219us) client_negotiation.cc:240] Negotiation successful
0806 09:33:30.762556 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":677,"thread_start_us":396,"threads_started":1}
I20260806 09:33:30.765136 14098 heartbeater.cc:347] Connected to a master server at 127.20.113.62:41393
I20260806 09:33:30.765447 14098 heartbeater.cc:464] Registering TS with master...
I20260806 09:33:30.766222 14098 heartbeater.cc:511] Master 127.20.113.62:41393 requested a full tablet report, sending...
I20260806 09:33:30.767674 13446 ts_manager.cc:194] Registered new tserver with Master: 8db53ae1e86248858c09c8aca3c43655 (127.20.113.3:41867)
I20260806 09:33:30.768621 13446 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.113.3@KRBTEST.COM'} at 127.20.113.3:38699
I20260806 09:33:30.772158 20932 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 09:33:30.789746 14102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:30.776731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49170 (local address 127.20.113.62:41393)
0806 09:33:30.776895 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0806 09:33:30.776898 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 09:33:30.777344 (+ 446us) server_negotiation.cc:407] Connection header received
0806 09:33:30.777552 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:33:30.777557 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:33:30.777623 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:33:30.777726 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:33:30.779206 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:30.779913 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:30.781452 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:30.781918 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:30.782061 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:30.784512 (+ 2451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:33:30.784517 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:33:30.784520 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:33:30.784551 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:33:30.786883 (+ 2332us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:33:30.787594 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:33:30.787602 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:33:30.787605 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:33:30.787729 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:33:30.788152 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:33:30.788159 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:33:30.788162 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:33:30.788550 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:33:30.788840 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0806 09:33:30.789201 (+ 361us) server_negotiation.cc:299] Negotiation successful
0806 09:33:30.789419 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 09:33:30.793488 13446 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49170:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 09:33:30.800350 13446 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 09:33:30.822746 14119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:30.811908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.2:46655 (local address 127.0.0.1:35526)
0806 09:33:30.812935 (+ 1027us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:30.812994 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0806 09:33:30.813214 (+ 220us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:33:30.814076 (+ 862us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:33:30.814085 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:33:30.814119 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:33:30.814513 (+ 394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:33:30.814528 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:33:30.816660 (+ 2132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:33:30.816665 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:33:30.819154 (+ 2489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:33:30.819173 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:33:30.821233 (+ 2060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:33:30.821239 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:33:30.821727 (+ 488us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:30.821908 (+ 181us) client_negotiation.cc:769] Sending connection context
0806 09:33:30.822098 (+ 190us) client_negotiation.cc:240] Negotiation successful
0806 09:33:30.822282 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":754,"mutex_wait_us":143,"thread_start_us":579,"threads_started":1}
I20260806 09:33:30.823499 14120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:30.812232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35526 (local address 127.20.113.2:46655)
0806 09:33:30.812983 (+ 751us) server_negotiation.cc:206] Beginning negotiation
0806 09:33:30.812994 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 09:33:30.813113 (+ 119us) server_negotiation.cc:407] Connection header received
0806 09:33:30.813512 (+ 399us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:33:30.813524 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:33:30.813817 (+ 293us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:33:30.814737 (+ 920us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:33:30.815032 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:30.816488 (+ 1456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:30.819341 (+ 2853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:30.821033 (+ 1692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:30.821167 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:30.821370 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0806 09:33:30.822995 (+ 1625us) server_negotiation.cc:299] Negotiation successful
0806 09:33:30.823175 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":588,"thread_start_us":438,"threads_started":1}
I20260806 09:33:30.825001 14125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:30.814501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.3:41867 (local address 127.0.0.1:47482)
0806 09:33:30.815515 (+ 1014us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:30.815548 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 09:33:30.815785 (+ 237us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:33:30.816832 (+ 1047us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:33:30.816836 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:33:30.816868 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:33:30.817219 (+ 351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:33:30.817227 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:33:30.819038 (+ 1811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:33:30.819046 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:33:30.821656 (+ 2610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:33:30.821675 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:33:30.824112 (+ 2437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:33:30.824117 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:33:30.824232 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:30.824252 (+ 20us) client_negotiation.cc:769] Sending connection context
0806 09:33:30.824413 (+ 161us) client_negotiation.cc:240] Negotiation successful
0806 09:33:30.824556 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":862,"mutex_wait_us":124,"thread_start_us":698,"threads_started":1}
I20260806 09:33:30.824996 14124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:30.814200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47482 (local address 127.20.113.3:41867)
0806 09:33:30.814988 (+ 788us) server_negotiation.cc:206] Beginning negotiation
0806 09:33:30.814999 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 09:33:30.815653 (+ 654us) server_negotiation.cc:407] Connection header received
0806 09:33:30.816106 (+ 453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:33:30.816117 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:33:30.816445 (+ 328us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:33:30.816620 (+ 175us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:33:30.817439 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:30.818874 (+ 1435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:30.821886 (+ 3012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:30.823590 (+ 1704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:30.823803 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:30.823968 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0806 09:33:30.824455 (+ 487us) server_negotiation.cc:299] Negotiation successful
0806 09:33:30.824637 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":558,"thread_start_us":468,"threads_started":1}
I20260806 09:33:30.825698 14121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:30.813273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.1:39175 (local address 127.0.0.1:45514)
0806 09:33:30.813821 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:30.813850 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0806 09:33:30.813985 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:33:30.817462 (+ 3477us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:33:30.817466 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:33:30.817491 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:33:30.817927 (+ 436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:33:30.817944 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:33:30.819810 (+ 1866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:33:30.819815 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:33:30.821700 (+ 1885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:33:30.821716 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:33:30.824886 (+ 3170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:33:30.824893 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:33:30.825069 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:30.825100 (+ 31us) client_negotiation.cc:769] Sending connection context
0806 09:33:30.825168 (+ 68us) client_negotiation.cc:240] Negotiation successful
0806 09:33:30.825298 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":424,"mutex_wait_us":133,"thread_start_us":270,"threads_started":1}
I20260806 09:33:30.827214 14122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:30.813692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45514 (local address 127.20.113.1:39175)
0806 09:33:30.816877 (+ 3185us) server_negotiation.cc:206] Beginning negotiation
0806 09:33:30.816887 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 09:33:30.816914 (+ 27us) server_negotiation.cc:407] Connection header received
0806 09:33:30.817006 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:33:30.817013 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:33:30.817226 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:33:30.818131 (+ 905us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:33:30.818361 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:30.819674 (+ 1313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:30.821934 (+ 2260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:30.824605 (+ 2671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:30.826422 (+ 1817us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:30.826650 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0806 09:33:30.826716 (+ 66us) server_negotiation.cc:299] Negotiation successful
0806 09:33:30.826885 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2991,"thread_start_us":2819,"threads_started":1}
I20260806 09:33:30.827653 13810 tablet_service.cc:1467] Processing CreateTablet for tablet e86a9c2bbae44931999f9929ea301451 (DEFAULT_TABLE table=test-table [id=df7022be1dfa4a5694e28165343bc8dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:33:30.828358 14005 tablet_service.cc:1467] Processing CreateTablet for tablet e86a9c2bbae44931999f9929ea301451 (DEFAULT_TABLE table=test-table [id=df7022be1dfa4a5694e28165343bc8dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:33:30.829804 14005 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e86a9c2bbae44931999f9929ea301451. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:33:30.829154 13810 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e86a9c2bbae44931999f9929ea301451. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:33:30.831138 13620 tablet_service.cc:1467] Processing CreateTablet for tablet e86a9c2bbae44931999f9929ea301451 (DEFAULT_TABLE table=test-table [id=df7022be1dfa4a5694e28165343bc8dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:33:30.832475 13620 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e86a9c2bbae44931999f9929ea301451. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:33:30.843484 14126 tablet_bootstrap.cc:492] T e86a9c2bbae44931999f9929ea301451 P 4faa2a9033fd4777adb64589c43b9df2: Bootstrap starting.
I20260806 09:33:30.846190 14128 tablet_bootstrap.cc:492] T e86a9c2bbae44931999f9929ea301451 P a696398c785a478b8810d2b956faa8b2: Bootstrap starting.
I20260806 09:33:30.847719 14126 tablet_bootstrap.cc:654] T e86a9c2bbae44931999f9929ea301451 P 4faa2a9033fd4777adb64589c43b9df2: Neither blocks nor log segments found. Creating new log.
I20260806 09:33:30.847895 14129 tablet_bootstrap.cc:492] T e86a9c2bbae44931999f9929ea301451 P 8db53ae1e86248858c09c8aca3c43655: Bootstrap starting.
I20260806 09:33:30.849236 14126 log.cc:824] T e86a9c2bbae44931999f9929ea301451 P 4faa2a9033fd4777adb64589c43b9df2: Log is configured to *not* fsync() on all Append() calls
I20260806 09:33:30.849800 14128 tablet_bootstrap.cc:654] T e86a9c2bbae44931999f9929ea301451 P a696398c785a478b8810d2b956faa8b2: Neither blocks nor log segments found. Creating new log.
I20260806 09:33:30.850907 14128 log.cc:824] T e86a9c2bbae44931999f9929ea301451 P a696398c785a478b8810d2b956faa8b2: Log is configured to *not* fsync() on all Append() calls
I20260806 09:33:30.851138 14129 tablet_bootstrap.cc:654] T e86a9c2bbae44931999f9929ea301451 P 8db53ae1e86248858c09c8aca3c43655: Neither blocks nor log segments found. Creating new log.
I20260806 09:33:30.852355 14126 tablet_bootstrap.cc:492] T e86a9c2bbae44931999f9929ea301451 P 4faa2a9033fd4777adb64589c43b9df2: No bootstrap required, opened a new log
I20260806 09:33:30.852458 14129 log.cc:824] T e86a9c2bbae44931999f9929ea301451 P 8db53ae1e86248858c09c8aca3c43655: Log is configured to *not* fsync() on all Append() calls
I20260806 09:33:30.852629 14126 ts_tablet_manager.cc:1397] T e86a9c2bbae44931999f9929ea301451 P 4faa2a9033fd4777adb64589c43b9df2: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.002s
I20260806 09:33:30.854313 14128 tablet_bootstrap.cc:492] T e86a9c2bbae44931999f9929ea301451 P a696398c785a478b8810d2b956faa8b2: No bootstrap required, opened a new log
I20260806 09:33:30.854564 14128 ts_tablet_manager.cc:1397] T e86a9c2bbae44931999f9929ea301451 P a696398c785a478b8810d2b956faa8b2: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260806 09:33:30.855301 14129 tablet_bootstrap.cc:492] T e86a9c2bbae44931999f9929ea301451 P 8db53ae1e86248858c09c8aca3c43655: No bootstrap required, opened a new log
I20260806 09:33:30.855548 14129 ts_tablet_manager.cc:1397] T e86a9c2bbae44931999f9929ea301451 P 8db53ae1e86248858c09c8aca3c43655: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260806 09:33:30.862968 14129 raft_consensus.cc:348] T e86a9c2bbae44931999f9929ea301451 P 8db53ae1e86248858c09c8aca3c43655 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a696398c785a478b8810d2b956faa8b2" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 46655 } } peers { permanent_uuid: "4faa2a9033fd4777adb64589c43b9df2" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 39175 } } peers { permanent_uuid: "8db53ae1e86248858c09c8aca3c43655" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 41867 } }
I20260806 09:33:30.863340 14129 raft_consensus.cc:374] T e86a9c2bbae44931999f9929ea301451 P 8db53ae1e86248858c09c8aca3c43655 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:33:30.863476 14129 raft_consensus.cc:729] T e86a9c2bbae44931999f9929ea301451 P 8db53ae1e86248858c09c8aca3c43655 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8db53ae1e86248858c09c8aca3c43655, State: Initialized, Role: FOLLOWER
I20260806 09:33:30.863363 14126 raft_consensus.cc:348] T e86a9c2bbae44931999f9929ea301451 P 4faa2a9033fd4777adb64589c43b9df2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a696398c785a478b8810d2b956faa8b2" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 46655 } } peers { permanent_uuid: "4faa2a9033fd4777adb64589c43b9df2" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 39175 } } peers { permanent_uuid: "8db53ae1e86248858c09c8aca3c43655" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 41867 } }
I20260806 09:33:30.863766 14126 raft_consensus.cc:374] T e86a9c2bbae44931999f9929ea301451 P 4faa2a9033fd4777adb64589c43b9df2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:33:30.863864 14126 raft_consensus.cc:729] T e86a9c2bbae44931999f9929ea301451 P 4faa2a9033fd4777adb64589c43b9df2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4faa2a9033fd4777adb64589c43b9df2, State: Initialized, Role: FOLLOWER
I20260806 09:33:30.864087 14129 consensus_queue.cc:260] T e86a9c2bbae44931999f9929ea301451 P 8db53ae1e86248858c09c8aca3c43655 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a696398c785a478b8810d2b956faa8b2" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 46655 } } peers { permanent_uuid: "4faa2a9033fd4777adb64589c43b9df2" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 39175 } } peers { permanent_uuid: "8db53ae1e86248858c09c8aca3c43655" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 41867 } }
I20260806 09:33:30.864547 14126 consensus_queue.cc:260] T e86a9c2bbae44931999f9929ea301451 P 4faa2a9033fd4777adb64589c43b9df2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a696398c785a478b8810d2b956faa8b2" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 46655 } } peers { permanent_uuid: "4faa2a9033fd4777adb64589c43b9df2" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 39175 } } peers { permanent_uuid: "8db53ae1e86248858c09c8aca3c43655" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 41867 } }
I20260806 09:33:30.865657 14128 raft_consensus.cc:348] T e86a9c2bbae44931999f9929ea301451 P a696398c785a478b8810d2b956faa8b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a696398c785a478b8810d2b956faa8b2" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 46655 } } peers { permanent_uuid: "4faa2a9033fd4777adb64589c43b9df2" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 39175 } } peers { permanent_uuid: "8db53ae1e86248858c09c8aca3c43655" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 41867 } }
I20260806 09:33:30.866168 14128 raft_consensus.cc:374] T e86a9c2bbae44931999f9929ea301451 P a696398c785a478b8810d2b956faa8b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:33:30.866309 14128 raft_consensus.cc:729] T e86a9c2bbae44931999f9929ea301451 P a696398c785a478b8810d2b956faa8b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a696398c785a478b8810d2b956faa8b2, State: Initialized, Role: FOLLOWER
I20260806 09:33:30.867616 14098 heartbeater.cc:503] Master 127.20.113.62:41393 was elected leader, sending a full tablet report...
I20260806 09:33:30.867769 14126 ts_tablet_manager.cc:1428] T e86a9c2bbae44931999f9929ea301451 P 4faa2a9033fd4777adb64589c43b9df2: Time spent starting tablet: real 0.015s user 0.012s sys 0.002s
I20260806 09:33:30.867511 14128 consensus_queue.cc:260] T e86a9c2bbae44931999f9929ea301451 P a696398c785a478b8810d2b956faa8b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a696398c785a478b8810d2b956faa8b2" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 46655 } } peers { permanent_uuid: "4faa2a9033fd4777adb64589c43b9df2" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 39175 } } peers { permanent_uuid: "8db53ae1e86248858c09c8aca3c43655" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 41867 } }
I20260806 09:33:30.869212 14129 ts_tablet_manager.cc:1428] T e86a9c2bbae44931999f9929ea301451 P 8db53ae1e86248858c09c8aca3c43655: Time spent starting tablet: real 0.014s user 0.004s sys 0.010s
I20260806 09:33:30.870074 13901 heartbeater.cc:503] Master 127.20.113.62:41393 was elected leader, sending a full tablet report...
I20260806 09:33:30.870705 14128 ts_tablet_manager.cc:1428] T e86a9c2bbae44931999f9929ea301451 P a696398c785a478b8810d2b956faa8b2: Time spent starting tablet: real 0.016s user 0.014s sys 0.002s
W20260806 09:33:30.914304 13904 tablet.cc:2369] T e86a9c2bbae44931999f9929ea301451 P a696398c785a478b8810d2b956faa8b2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 09:33:30.963526 14136 raft_consensus.cc:482] T e86a9c2bbae44931999f9929ea301451 P a696398c785a478b8810d2b956faa8b2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 09:33:30.963845 14136 raft_consensus.cc:504] T e86a9c2bbae44931999f9929ea301451 P a696398c785a478b8810d2b956faa8b2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a696398c785a478b8810d2b956faa8b2" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 46655 } } peers { permanent_uuid: "4faa2a9033fd4777adb64589c43b9df2" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 39175 } } peers { permanent_uuid: "8db53ae1e86248858c09c8aca3c43655" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 41867 } }
I20260806 09:33:30.965513 14136 leader_election.cc:288] T e86a9c2bbae44931999f9929ea301451 P a696398c785a478b8810d2b956faa8b2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4faa2a9033fd4777adb64589c43b9df2 (127.20.113.1:39175), 8db53ae1e86248858c09c8aca3c43655 (127.20.113.3:41867)
I20260806 09:33:30.971354 14134 raft_consensus.cc:482] T e86a9c2bbae44931999f9929ea301451 P 8db53ae1e86248858c09c8aca3c43655 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 09:33:30.971697 14134 raft_consensus.cc:504] T e86a9c2bbae44931999f9929ea301451 P 8db53ae1e86248858c09c8aca3c43655 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a696398c785a478b8810d2b956faa8b2" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 46655 } } peers { permanent_uuid: "4faa2a9033fd4777adb64589c43b9df2" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 39175 } } peers { permanent_uuid: "8db53ae1e86248858c09c8aca3c43655" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 41867 } }
I20260806 09:33:30.973891 14124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:30.967131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.2:57973 (local address 127.20.113.3:41867)
0806 09:33:30.967333 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0806 09:33:30.967339 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:33:30.967871 (+ 532us) server_negotiation.cc:407] Connection header received
0806 09:33:30.968057 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:33:30.968066 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:33:30.968151 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:33:30.968256 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:33:30.968939 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:30.969853 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:30.971837 (+ 1984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:30.972832 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:30.973043 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:30.973116 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 09:33:30.973489 (+ 373us) server_negotiation.cc:299] Negotiation successful
0806 09:33:30.973620 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 09:33:30.974138 14122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:30.966199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.2:51835 (local address 127.20.113.1:39175)
0806 09:33:30.966470 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0806 09:33:30.966477 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:33:30.966906 (+ 429us) server_negotiation.cc:407] Connection header received
0806 09:33:30.967233 (+ 327us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:33:30.967239 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:33:30.967338 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:33:30.967410 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:33:30.968393 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:30.969357 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:30.971657 (+ 2300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:30.972940 (+ 1283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:30.973099 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:30.973165 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 09:33:30.973762 (+ 597us) server_negotiation.cc:299] Negotiation successful
0806 09:33:30.973893 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260806 09:33:30.974385 14138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:30.966876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.3:41867 (local address 127.20.113.2:57973)
0806 09:33:30.967723 (+ 847us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:30.967756 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 09:33:30.967915 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:33:30.968279 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:33:30.968282 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:33:30.968350 (+ 68us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:33:30.968777 (+ 427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:33:30.968792 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:33:30.969999 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:33:30.970004 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:33:30.971671 (+ 1667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:33:30.971685 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:33:30.973067 (+ 1382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:33:30.973074 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:33:30.973280 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:30.973337 (+ 57us) client_negotiation.cc:769] Sending connection context
0806 09:33:30.973436 (+ 99us) client_negotiation.cc:240] Negotiation successful
0806 09:33:30.973573 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":724,"mutex_wait_us":39,"thread_start_us":529,"threads_started":1}
I20260806 09:33:30.974885 14134 leader_election.cc:288] T e86a9c2bbae44931999f9929ea301451 P 8db53ae1e86248858c09c8aca3c43655 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a696398c785a478b8810d2b956faa8b2 (127.20.113.2:46655), 4faa2a9033fd4777adb64589c43b9df2 (127.20.113.1:39175)
I20260806 09:33:30.975113 14137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:30.965903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.1:39175 (local address 127.20.113.2:51835)
0806 09:33:30.966792 (+ 889us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:30.966812 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 09:33:30.967051 (+ 239us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:33:30.967646 (+ 595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:33:30.967651 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:33:30.967680 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:33:30.968220 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:33:30.968237 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:33:30.969496 (+ 1259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:33:30.969502 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:33:30.971418 (+ 1916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:33:30.971435 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:33:30.973277 (+ 1842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:33:30.973283 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:33:30.973542 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:30.973569 (+ 27us) client_negotiation.cc:769] Sending connection context
0806 09:33:30.974420 (+ 851us) client_negotiation.cc:240] Negotiation successful
0806 09:33:30.974626 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":699,"mutex_wait_us":84,"spinlock_wait_cycles":24448,"thread_start_us":450,"threads_started":1}
I20260806 09:33:30.975878 14025 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e86a9c2bbae44931999f9929ea301451" candidate_uuid: "a696398c785a478b8810d2b956faa8b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8db53ae1e86248858c09c8aca3c43655" is_pre_election: true
I20260806 09:33:30.976438 14025 raft_consensus.cc:2454] T e86a9c2bbae44931999f9929ea301451 P 8db53ae1e86248858c09c8aca3c43655 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a696398c785a478b8810d2b956faa8b2 in term 0.
I20260806 09:33:30.976423 13646 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e86a9c2bbae44931999f9929ea301451" candidate_uuid: "a696398c785a478b8810d2b956faa8b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4faa2a9033fd4777adb64589c43b9df2" is_pre_election: true
I20260806 09:33:30.977036 13646 raft_consensus.cc:2454] T e86a9c2bbae44931999f9929ea301451 P 4faa2a9033fd4777adb64589c43b9df2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a696398c785a478b8810d2b956faa8b2 in term 0.
I20260806 09:33:30.977293 13737 leader_election.cc:302] T e86a9c2bbae44931999f9929ea301451 P a696398c785a478b8810d2b956faa8b2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8db53ae1e86248858c09c8aca3c43655, a696398c785a478b8810d2b956faa8b2; no voters:
I20260806 09:33:30.977715 14136 raft_consensus.cc:2781] T e86a9c2bbae44931999f9929ea301451 P a696398c785a478b8810d2b956faa8b2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 09:33:30.977881 14136 raft_consensus.cc:482] T e86a9c2bbae44931999f9929ea301451 P a696398c785a478b8810d2b956faa8b2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 09:33:30.978008 14136 raft_consensus.cc:3037] T e86a9c2bbae44931999f9929ea301451 P a696398c785a478b8810d2b956faa8b2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:33:30.981037 14136 raft_consensus.cc:504] T e86a9c2bbae44931999f9929ea301451 P a696398c785a478b8810d2b956faa8b2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a696398c785a478b8810d2b956faa8b2" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 46655 } } peers { permanent_uuid: "4faa2a9033fd4777adb64589c43b9df2" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 39175 } } peers { permanent_uuid: "8db53ae1e86248858c09c8aca3c43655" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 41867 } }
I20260806 09:33:30.981725 14136 leader_election.cc:288] T e86a9c2bbae44931999f9929ea301451 P a696398c785a478b8810d2b956faa8b2 [CANDIDATE]: Term 1 election: Requested vote from peers 4faa2a9033fd4777adb64589c43b9df2 (127.20.113.1:39175), 8db53ae1e86248858c09c8aca3c43655 (127.20.113.3:41867)
I20260806 09:33:30.982224 14105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:30.974223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.2:46655 (local address 127.20.113.3:34635)
0806 09:33:30.974540 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:30.974559 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 09:33:30.974676 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:33:30.975198 (+ 522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:33:30.975205 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:33:30.975231 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:33:30.975426 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:33:30.975437 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:33:30.977354 (+ 1917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:33:30.977363 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:33:30.979507 (+ 2144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:33:30.979524 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:33:30.981385 (+ 1861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:33:30.981397 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:33:30.981626 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:30.981662 (+ 36us) client_negotiation.cc:769] Sending connection context
0806 09:33:30.981801 (+ 139us) client_negotiation.cc:240] Negotiation successful
0806 09:33:30.981948 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":203}
I20260806 09:33:30.982272 14120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:30.974383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.3:34635 (local address 127.20.113.2:46655)
0806 09:33:30.974596 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0806 09:33:30.974603 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:33:30.974621 (+ 18us) server_negotiation.cc:407] Connection header received
0806 09:33:30.974860 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:33:30.974866 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:33:30.974946 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:33:30.975032 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:33:30.975720 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:30.977144 (+ 1424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:30.979679 (+ 2535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:30.981115 (+ 1436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:30.981270 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:30.981340 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 09:33:30.981936 (+ 596us) server_negotiation.cc:299] Negotiation successful
0806 09:33:30.982058 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260806 09:33:30.982455 13646 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e86a9c2bbae44931999f9929ea301451" candidate_uuid: "a696398c785a478b8810d2b956faa8b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4faa2a9033fd4777adb64589c43b9df2"
I20260806 09:33:30.982666 13646 raft_consensus.cc:3037] T e86a9c2bbae44931999f9929ea301451 P 4faa2a9033fd4777adb64589c43b9df2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:33:30.982918 14025 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e86a9c2bbae44931999f9929ea301451" candidate_uuid: "a696398c785a478b8810d2b956faa8b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8db53ae1e86248858c09c8aca3c43655"
I20260806 09:33:30.983141 14025 raft_consensus.cc:3037] T e86a9c2bbae44931999f9929ea301451 P 8db53ae1e86248858c09c8aca3c43655 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:33:30.983129 13840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e86a9c2bbae44931999f9929ea301451" candidate_uuid: "8db53ae1e86248858c09c8aca3c43655" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a696398c785a478b8810d2b956faa8b2" is_pre_election: true
I20260806 09:33:30.983649 13840 raft_consensus.cc:2379] T e86a9c2bbae44931999f9929ea301451 P a696398c785a478b8810d2b956faa8b2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8db53ae1e86248858c09c8aca3c43655 in current term 1: Already voted for candidate a696398c785a478b8810d2b956faa8b2 in this term.
I20260806 09:33:30.985880 14122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:30.975815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.3:42009 (local address 127.20.113.1:39175)
0806 09:33:30.977773 (+ 1958us) server_negotiation.cc:206] Beginning negotiation
0806 09:33:30.977779 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:33:30.978479 (+ 700us) server_negotiation.cc:407] Connection header received
0806 09:33:30.978671 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:33:30.978675 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:33:30.978725 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:33:30.978820 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:33:30.979431 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:30.980520 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:30.983450 (+ 2930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:30.984871 (+ 1421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:30.985076 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:30.985152 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 09:33:30.985498 (+ 346us) server_negotiation.cc:299] Negotiation successful
0806 09:33:30.985625 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1850}
I20260806 09:33:30.985924 14140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:30.975310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.1:39175 (local address 127.20.113.3:42009)
0806 09:33:30.978378 (+ 3068us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:30.978393 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 09:33:30.978535 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:33:30.978835 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:33:30.978839 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:33:30.978873 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:33:30.979254 (+ 381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:33:30.979262 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:33:30.980693 (+ 1431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:33:30.980698 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:33:30.983243 (+ 2545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:33:30.983261 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:33:30.985124 (+ 1863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:33:30.985134 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:33:30.985322 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:30.985360 (+ 38us) client_negotiation.cc:769] Sending connection context
0806 09:33:30.985456 (+ 96us) client_negotiation.cc:240] Negotiation successful
0806 09:33:30.985600 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2945,"thread_start_us":2797,"threads_started":1}
I20260806 09:33:30.986497 13646 raft_consensus.cc:2454] T e86a9c2bbae44931999f9929ea301451 P 4faa2a9033fd4777adb64589c43b9df2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a696398c785a478b8810d2b956faa8b2 in term 1.
I20260806 09:33:30.986622 14025 raft_consensus.cc:2454] T e86a9c2bbae44931999f9929ea301451 P 8db53ae1e86248858c09c8aca3c43655 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a696398c785a478b8810d2b956faa8b2 in term 1.
I20260806 09:33:30.986804 13645 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e86a9c2bbae44931999f9929ea301451" candidate_uuid: "8db53ae1e86248858c09c8aca3c43655" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4faa2a9033fd4777adb64589c43b9df2" is_pre_election: true
I20260806 09:33:30.986984 13737 leader_election.cc:302] T e86a9c2bbae44931999f9929ea301451 P a696398c785a478b8810d2b956faa8b2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4faa2a9033fd4777adb64589c43b9df2, a696398c785a478b8810d2b956faa8b2; no voters:
I20260806 09:33:30.987249 13645 raft_consensus.cc:2379] T e86a9c2bbae44931999f9929ea301451 P 4faa2a9033fd4777adb64589c43b9df2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8db53ae1e86248858c09c8aca3c43655 in current term 1: Already voted for candidate a696398c785a478b8810d2b956faa8b2 in this term.
I20260806 09:33:30.987306 14136 raft_consensus.cc:2781] T e86a9c2bbae44931999f9929ea301451 P a696398c785a478b8810d2b956faa8b2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:33:30.987847 14136 raft_consensus.cc:686] T e86a9c2bbae44931999f9929ea301451 P a696398c785a478b8810d2b956faa8b2 [term 1 LEADER]: Becoming Leader. State: Replica: a696398c785a478b8810d2b956faa8b2, State: Running, Role: LEADER
I20260806 09:33:30.987953 13930 leader_election.cc:302] T e86a9c2bbae44931999f9929ea301451 P 8db53ae1e86248858c09c8aca3c43655 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8db53ae1e86248858c09c8aca3c43655; no voters: 4faa2a9033fd4777adb64589c43b9df2, a696398c785a478b8810d2b956faa8b2
I20260806 09:33:30.988413 14134 raft_consensus.cc:2726] T e86a9c2bbae44931999f9929ea301451 P 8db53ae1e86248858c09c8aca3c43655 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 09:33:30.988471 14136 consensus_queue.cc:237] T e86a9c2bbae44931999f9929ea301451 P a696398c785a478b8810d2b956faa8b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a696398c785a478b8810d2b956faa8b2" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 46655 } } peers { permanent_uuid: "4faa2a9033fd4777adb64589c43b9df2" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 39175 } } peers { permanent_uuid: "8db53ae1e86248858c09c8aca3c43655" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 41867 } }
W20260806 09:33:30.992895 14101 tablet.cc:2369] T e86a9c2bbae44931999f9929ea301451 P 8db53ae1e86248858c09c8aca3c43655: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 09:33:30.993397 13445 catalog_manager.cc:5495] T e86a9c2bbae44931999f9929ea301451 P a696398c785a478b8810d2b956faa8b2 reported cstate change: term changed from 0 to 1, leader changed from <none> to a696398c785a478b8810d2b956faa8b2 (127.20.113.2). New cstate: current_term: 1 leader_uuid: "a696398c785a478b8810d2b956faa8b2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a696398c785a478b8810d2b956faa8b2" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 46655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4faa2a9033fd4777adb64589c43b9df2" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 39175 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8db53ae1e86248858c09c8aca3c43655" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 41867 } health_report { overall_health: UNKNOWN } } }
I20260806 09:33:31.018409 14120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:31.012608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35542 (local address 127.20.113.2:46655)
0806 09:33:31.012842 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0806 09:33:31.012849 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:33:31.012868 (+ 19us) server_negotiation.cc:407] Connection header received
0806 09:33:31.013132 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:33:31.013136 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:33:31.013230 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:33:31.013403 (+ 173us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 09:33:31.013821 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:31.014574 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:31.016017 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:31.016531 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:31.016669 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:31.017224 (+ 555us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 09:33:31.017752 (+ 528us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 09:33:31.017925 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0806 09:33:31.018004 (+ 79us) server_negotiation.cc:299] Negotiation successful
0806 09:33:31.018087 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260806 09:33:31.026925 13645 raft_consensus.cc:1261] T e86a9c2bbae44931999f9929ea301451 P 4faa2a9033fd4777adb64589c43b9df2 [term 1 FOLLOWER]: Refusing update from remote peer a696398c785a478b8810d2b956faa8b2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 09:33:31.026932 14025 raft_consensus.cc:1261] T e86a9c2bbae44931999f9929ea301451 P 8db53ae1e86248858c09c8aca3c43655 [term 1 FOLLOWER]: Refusing update from remote peer a696398c785a478b8810d2b956faa8b2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 09:33:31.027973 14142 consensus_queue.cc:1035] T e86a9c2bbae44931999f9929ea301451 P a696398c785a478b8810d2b956faa8b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4faa2a9033fd4777adb64589c43b9df2" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 39175 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 09:33:31.028429 14136 consensus_queue.cc:1035] T e86a9c2bbae44931999f9929ea301451 P a696398c785a478b8810d2b956faa8b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8db53ae1e86248858c09c8aca3c43655" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 41867 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 09:33:31.055637 13446 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49170:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 09:33:31.055967 13446 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49170:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 09:33:31.059986 13446 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e1ebfd4144014ce5975e107f5d969261: Sending DeleteTablet for 3 replicas of tablet e86a9c2bbae44931999f9929ea301451
I20260806 09:33:31.061278 13810 tablet_service.cc:1514] Processing DeleteTablet for tablet e86a9c2bbae44931999f9929ea301451 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:33:31 UTC) from {username='kudu', principal='kudu/127.20.113.62@KRBTEST.COM'} at 127.0.0.1:35526
I20260806 09:33:31.061496 13620 tablet_service.cc:1514] Processing DeleteTablet for tablet e86a9c2bbae44931999f9929ea301451 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:33:31 UTC) from {username='kudu', principal='kudu/127.20.113.62@KRBTEST.COM'} at 127.0.0.1:45514
I20260806 09:33:31.062390 14162 tablet_replica.cc:330] T e86a9c2bbae44931999f9929ea301451 P a696398c785a478b8810d2b956faa8b2: stopping tablet replica
I20260806 09:33:31.062532 14005 tablet_service.cc:1514] Processing DeleteTablet for tablet e86a9c2bbae44931999f9929ea301451 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:33:31 UTC) from {username='kudu', principal='kudu/127.20.113.62@KRBTEST.COM'} at 127.0.0.1:47482
I20260806 09:33:31.062973 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 13525
I20260806 09:33:31.063100 14162 raft_consensus.cc:2229] T e86a9c2bbae44931999f9929ea301451 P a696398c785a478b8810d2b956faa8b2 [term 1 LEADER]: Raft consensus shutting down.
I20260806 09:33:31.063884 14162 raft_consensus.cc:2258] T e86a9c2bbae44931999f9929ea301451 P a696398c785a478b8810d2b956faa8b2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 09:33:31.066776 14162 ts_tablet_manager.cc:1905] T e86a9c2bbae44931999f9929ea301451 P a696398c785a478b8810d2b956faa8b2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 09:33:31.074633 14162 ts_tablet_manager.cc:1918] T e86a9c2bbae44931999f9929ea301451 P a696398c785a478b8810d2b956faa8b2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 09:33:31.074919 14162 log.cc:1196] T e86a9c2bbae44931999f9929ea301451 P a696398c785a478b8810d2b956faa8b2: Deleting WAL directory at /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786008495763054-20932-0/minicluster-data/ts-1/wal/wals/e86a9c2bbae44931999f9929ea301451
I20260806 09:33:31.075636 14162 ts_tablet_manager.cc:1939] T e86a9c2bbae44931999f9929ea301451 P a696398c785a478b8810d2b956faa8b2: Deleting consensus metadata
I20260806 09:33:31.077677 13419 catalog_manager.cc:4841] TS a696398c785a478b8810d2b956faa8b2 (127.20.113.2:46655): tablet e86a9c2bbae44931999f9929ea301451 (table test-table [id=df7022be1dfa4a5694e28165343bc8dc]) successfully deleted
I20260806 09:33:31.079811 14164 tablet_replica.cc:330] T e86a9c2bbae44931999f9929ea301451 P 8db53ae1e86248858c09c8aca3c43655: stopping tablet replica
I20260806 09:33:31.080485 14164 raft_consensus.cc:2229] T e86a9c2bbae44931999f9929ea301451 P 8db53ae1e86248858c09c8aca3c43655 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 09:33:31.081141 14164 raft_consensus.cc:2258] T e86a9c2bbae44931999f9929ea301451 P 8db53ae1e86248858c09c8aca3c43655 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 09:33:31.083428 14163 tablet_replica.cc:330] T e86a9c2bbae44931999f9929ea301451 P 4faa2a9033fd4777adb64589c43b9df2: stopping tablet replica
I20260806 09:33:31.083899 14163 raft_consensus.cc:2229] T e86a9c2bbae44931999f9929ea301451 P 4faa2a9033fd4777adb64589c43b9df2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 09:33:31.084447 14163 raft_consensus.cc:2258] T e86a9c2bbae44931999f9929ea301451 P 4faa2a9033fd4777adb64589c43b9df2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 09:33:31.085968 14164 ts_tablet_manager.cc:1905] T e86a9c2bbae44931999f9929ea301451 P 8db53ae1e86248858c09c8aca3c43655: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 09:33:31.086570 14163 ts_tablet_manager.cc:1905] T e86a9c2bbae44931999f9929ea301451 P 4faa2a9033fd4777adb64589c43b9df2: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 06 09:33:31 dist-test-slave-45dx krb5kdc[13377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008808, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.113.1@KRBTEST.COM
I20260806 09:33:31.093827 14163 ts_tablet_manager.cc:1918] T e86a9c2bbae44931999f9929ea301451 P 4faa2a9033fd4777adb64589c43b9df2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 09:33:31.093815 14164 ts_tablet_manager.cc:1918] T e86a9c2bbae44931999f9929ea301451 P 8db53ae1e86248858c09c8aca3c43655: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 09:33:31.093999 14163 log.cc:1196] T e86a9c2bbae44931999f9929ea301451 P 4faa2a9033fd4777adb64589c43b9df2: Deleting WAL directory at /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786008495763054-20932-0/minicluster-data/ts-0/wal/wals/e86a9c2bbae44931999f9929ea301451
I20260806 09:33:31.094055 14164 log.cc:1196] T e86a9c2bbae44931999f9929ea301451 P 8db53ae1e86248858c09c8aca3c43655: Deleting WAL directory at /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786008495763054-20932-0/minicluster-data/ts-2/wal/wals/e86a9c2bbae44931999f9929ea301451
I20260806 09:33:31.094637 14163 ts_tablet_manager.cc:1939] T e86a9c2bbae44931999f9929ea301451 P 4faa2a9033fd4777adb64589c43b9df2: Deleting consensus metadata
I20260806 09:33:31.094630 14164 ts_tablet_manager.cc:1939] T e86a9c2bbae44931999f9929ea301451 P 8db53ae1e86248858c09c8aca3c43655: Deleting consensus metadata
I20260806 09:33:31.096501 13419 catalog_manager.cc:4841] TS 4faa2a9033fd4777adb64589c43b9df2 (127.20.113.1:39175): tablet e86a9c2bbae44931999f9929ea301451 (table test-table [id=df7022be1dfa4a5694e28165343bc8dc]) successfully deleted
I20260806 09:33:31.096859 13419 catalog_manager.cc:4841] TS 8db53ae1e86248858c09c8aca3c43655 (127.20.113.3:41867): tablet e86a9c2bbae44931999f9929ea301451 (table test-table [id=df7022be1dfa4a5694e28165343bc8dc]) successfully deleted
I20260806 09:33:31.098590 14122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:31.064981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45518 (local address 127.20.113.1:39175)
0806 09:33:31.065173 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0806 09:33:31.065181 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 09:33:31.075596 (+ 10415us) server_negotiation.cc:407] Connection header received
0806 09:33:31.076118 (+ 522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:33:31.076127 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:33:31.076239 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:33:31.076388 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:33:31.079277 (+ 2889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:31.080655 (+ 1378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:31.083774 (+ 3119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:31.084501 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:31.084631 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:31.090902 (+ 6271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:33:31.090921 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:33:31.090940 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:33:31.091002 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:33:31.095255 (+ 4253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:33:31.096460 (+ 1205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:33:31.096469 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:33:31.096473 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:33:31.096652 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:33:31.097154 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:33:31.097158 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:33:31.097160 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:33:31.097520 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:33:31.097738 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0806 09:33:31.098133 (+ 395us) server_negotiation.cc:299] Negotiation successful
0806 09:33:31.098293 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 09:33:31.099367 13702 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:33:31.225551 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 13525
I20260806 09:33:31.257711 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 13713
Aug 06 09:33:31 dist-test-slave-45dx krb5kdc[13377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008808, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.113.2@KRBTEST.COM
I20260806 09:33:31.281925 14120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:31.258654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35554 (local address 127.20.113.2:46655)
0806 09:33:31.258884 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0806 09:33:31.258891 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:33:31.258911 (+ 20us) server_negotiation.cc:407] Connection header received
0806 09:33:31.258985 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:33:31.258989 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:33:31.259053 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:33:31.259179 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:33:31.260671 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:31.261801 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:31.263813 (+ 2012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:31.264583 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:31.264722 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:31.271169 (+ 6447us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:33:31.271186 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:33:31.271209 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:33:31.271264 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:33:31.278107 (+ 6843us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:33:31.279062 (+ 955us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:33:31.279075 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:33:31.279081 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:33:31.279264 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:33:31.279743 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:33:31.279749 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:33:31.279752 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:33:31.280446 (+ 694us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:33:31.280785 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0806 09:33:31.281095 (+ 310us) server_negotiation.cc:299] Negotiation successful
0806 09:33:31.281463 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 09:33:31.282930 13896 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:33:31.408717 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 13713
I20260806 09:33:31.438104 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 13911
Aug 06 09:33:31 dist-test-slave-45dx krb5kdc[13377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008808, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.113.3@KRBTEST.COM
I20260806 09:33:31.453059 14124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:31.438828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47488 (local address 127.20.113.3:41867)
0806 09:33:31.438998 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0806 09:33:31.439002 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 09:33:31.439076 (+ 74us) server_negotiation.cc:407] Connection header received
0806 09:33:31.439223 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:33:31.439227 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:33:31.439304 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:33:31.439389 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:33:31.440725 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:31.441437 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:31.442720 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:31.443122 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:31.443212 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:31.447568 (+ 4356us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:33:31.447581 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:33:31.447597 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:33:31.447629 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:33:31.450494 (+ 2865us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:33:31.451148 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:33:31.451155 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:33:31.451157 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:33:31.451294 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:33:31.451662 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:33:31.451665 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:33:31.451667 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:33:31.452043 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:33:31.452276 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0806 09:33:31.452585 (+ 309us) server_negotiation.cc:299] Negotiation successful
0806 09:33:31.452779 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 09:33:31.453683 14090 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:33:31.544409 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 13911
I20260806 09:33:31.567859 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 13392
I20260806 09:33:31.569111 13482 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:33:31.678138 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 13392
2026-08-06T09:33:31Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (4839 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-06T09:33:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T09:33:31Z Disabled control of system clock
I20260806 09:33:31.785243 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786008495763054-20932-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786008495763054-20932-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786008495763054-20932-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.113.62:35037
--webserver_interface=127.20.113.62
--webserver_port=0
--builtin_ntp_servers=127.20.113.20:36275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.113.62:35037
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 09:33:32.134327 14178 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:33:32.134614 14178 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:33:32.134697 14178 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:33:32.141819 14178 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 09:33:32.141927 14178 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:33:32.141965 14178 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:33:32.142024 14178 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 09:33:32.142069 14178 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 09:33:32.151044 14178 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:36275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786008495763054-20932-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786008495763054-20932-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.113.62:35037
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.113.62:35037
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.113.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786008495763054-20932-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:33:32.152261 14178 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:33:32.154095 14178 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:33:32.166208 14189 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:33:32.166337 14193 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:33:32.166326 14188 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:33:32.167104 14178 server_base.cc:1034] running on GCE node
I20260806 09:33:32.168035 14178 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:33:32.169554 14178 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:33:32.170796 14178 hybrid_clock.cc:648] HybridClock initialized: now 1786008812170746 us; error 37 us; skew 500 ppm
I20260806 09:33:32.171205 14178 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 09:33:32.173667 14178 webserver.cc:460] Webserver started at http://127.20.113.62:36743/ using document root <none> and password file <none>
I20260806 09:33:32.174386 14178 fs_manager.cc:359] Metadata directory not provided
I20260806 09:33:32.174522 14178 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:33:32.174860 14178 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:33:32.178043 14178 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786008495763054-20932-0/minicluster-data/master-0/data/instance:
uuid: "fef67d497b9646ce8fdcfb3d2727bed4"
format_stamp: "Formatted at 2026-08-06 09:33:32 on dist-test-slave-45dx"
server_key: "9cb147770569642f9e7705fa68b01a13"
server_key_iv: "81335b64b23af0b1e30b02ffe8369784"
server_key_version: "encryptionkey@0"
I20260806 09:33:32.178933 14178 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786008495763054-20932-0/minicluster-data/master-0/wal/instance:
uuid: "fef67d497b9646ce8fdcfb3d2727bed4"
format_stamp: "Formatted at 2026-08-06 09:33:32 on dist-test-slave-45dx"
server_key: "9cb147770569642f9e7705fa68b01a13"
server_key_iv: "81335b64b23af0b1e30b02ffe8369784"
server_key_version: "encryptionkey@0"
I20260806 09:33:32.184531 14178 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260806 09:33:32.188117 14200 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:33:32.189489 14178 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 09:33:32.189708 14178 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786008495763054-20932-0/minicluster-data/master-0/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786008495763054-20932-0/minicluster-data/master-0/wal
uuid: "fef67d497b9646ce8fdcfb3d2727bed4"
format_stamp: "Formatted at 2026-08-06 09:33:32 on dist-test-slave-45dx"
server_key: "9cb147770569642f9e7705fa68b01a13"
server_key_iv: "81335b64b23af0b1e30b02ffe8369784"
server_key_version: "encryptionkey@0"
I20260806 09:33:32.189965 14178 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786008495763054-20932-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786008495763054-20932-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786008495763054-20932-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:33:32.212073 14178 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:33:32.212888 14178 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:33:32.213125 14178 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:33:32.231484 14178 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.62:35037
I20260806 09:33:32.231508 14270 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.62:35037 every 8 connection(s)
I20260806 09:33:32.232832 14178 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
I20260806 09:33:32.236726 14271 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:33:32.236832 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 14178
I20260806 09:33:32.237061 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786008495763054-20932-0/minicluster-data/master-0/wal/instance
I20260806 09:33:32.237534 20932 external_mini_cluster.cc:1428] Setting key b69b6d5d2f434e05b45d2fd0429a3039
I20260806 09:33:32.250245 14271 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fef67d497b9646ce8fdcfb3d2727bed4: Bootstrap starting.
I20260806 09:33:32.251818 14276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:32.239381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57526 (local address 127.20.113.62:35037)
0806 09:33:32.240498 (+ 1117us) server_negotiation.cc:206] Beginning negotiation
0806 09:33:32.240508 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 09:33:32.240552 (+ 44us) server_negotiation.cc:407] Connection header received
0806 09:33:32.242910 (+ 2358us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:33:32.242932 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:33:32.243368 (+ 436us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:33:32.243664 (+ 296us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:33:32.244384 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:32.245600 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:32.247630 (+ 2030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:32.248231 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:32.248419 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:32.249564 (+ 1145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:33:32.249571 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:33:32.249583 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:33:32.250095 (+ 512us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:33:32.250163 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:33:32.250182 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:33:32.250296 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0806 09:33:32.250591 (+ 295us) server_negotiation.cc:299] Negotiation successful
0806 09:33:32.250922 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":630,"thread_start_us":406,"threads_started":1}
I20260806 09:33:32.253595 14271 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fef67d497b9646ce8fdcfb3d2727bed4: Neither blocks nor log segments found. Creating new log.
I20260806 09:33:32.254789 14271 log.cc:824] T 00000000000000000000000000000000 P fef67d497b9646ce8fdcfb3d2727bed4: Log is configured to *not* fsync() on all Append() calls
I20260806 09:33:32.257464 14271 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fef67d497b9646ce8fdcfb3d2727bed4: No bootstrap required, opened a new log
I20260806 09:33:32.266022 14271 raft_consensus.cc:348] T 00000000000000000000000000000000 P fef67d497b9646ce8fdcfb3d2727bed4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fef67d497b9646ce8fdcfb3d2727bed4" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 35037 } }
I20260806 09:33:32.266520 14271 raft_consensus.cc:374] T 00000000000000000000000000000000 P fef67d497b9646ce8fdcfb3d2727bed4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:33:32.266637 14271 raft_consensus.cc:729] T 00000000000000000000000000000000 P fef67d497b9646ce8fdcfb3d2727bed4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fef67d497b9646ce8fdcfb3d2727bed4, State: Initialized, Role: FOLLOWER
I20260806 09:33:32.267293 14271 consensus_queue.cc:260] T 00000000000000000000000000000000 P fef67d497b9646ce8fdcfb3d2727bed4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fef67d497b9646ce8fdcfb3d2727bed4" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 35037 } }
I20260806 09:33:32.267577 14271 raft_consensus.cc:388] T 00000000000000000000000000000000 P fef67d497b9646ce8fdcfb3d2727bed4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 09:33:32.267756 14271 raft_consensus.cc:482] T 00000000000000000000000000000000 P fef67d497b9646ce8fdcfb3d2727bed4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 09:33:32.267958 14271 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fef67d497b9646ce8fdcfb3d2727bed4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:33:32.271201 14271 raft_consensus.cc:504] T 00000000000000000000000000000000 P fef67d497b9646ce8fdcfb3d2727bed4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fef67d497b9646ce8fdcfb3d2727bed4" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 35037 } }
I20260806 09:33:32.271704 14271 leader_election.cc:302] T 00000000000000000000000000000000 P fef67d497b9646ce8fdcfb3d2727bed4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fef67d497b9646ce8fdcfb3d2727bed4; no voters:
I20260806 09:33:32.272282 14271 leader_election.cc:288] T 00000000000000000000000000000000 P fef67d497b9646ce8fdcfb3d2727bed4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 09:33:32.272502 14279 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fef67d497b9646ce8fdcfb3d2727bed4 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:33:32.273233 14279 raft_consensus.cc:686] T 00000000000000000000000000000000 P fef67d497b9646ce8fdcfb3d2727bed4 [term 1 LEADER]: Becoming Leader. State: Replica: fef67d497b9646ce8fdcfb3d2727bed4, State: Running, Role: LEADER
I20260806 09:33:32.273694 14279 consensus_queue.cc:237] T 00000000000000000000000000000000 P fef67d497b9646ce8fdcfb3d2727bed4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fef67d497b9646ce8fdcfb3d2727bed4" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 35037 } }
I20260806 09:33:32.274358 14271 sys_catalog.cc:564] T 00000000000000000000000000000000 P fef67d497b9646ce8fdcfb3d2727bed4 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 09:33:32.276825 14282 sys_catalog.cc:455] T 00000000000000000000000000000000 P fef67d497b9646ce8fdcfb3d2727bed4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fef67d497b9646ce8fdcfb3d2727bed4. Latest consensus state: current_term: 1 leader_uuid: "fef67d497b9646ce8fdcfb3d2727bed4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fef67d497b9646ce8fdcfb3d2727bed4" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 35037 } } }
I20260806 09:33:32.277272 14282 sys_catalog.cc:458] T 00000000000000000000000000000000 P fef67d497b9646ce8fdcfb3d2727bed4 [sys.catalog]: This master's current role is: LEADER
I20260806 09:33:32.276881 14281 sys_catalog.cc:455] T 00000000000000000000000000000000 P fef67d497b9646ce8fdcfb3d2727bed4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fef67d497b9646ce8fdcfb3d2727bed4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fef67d497b9646ce8fdcfb3d2727bed4" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 35037 } } }
I20260806 09:33:32.277943 14281 sys_catalog.cc:458] T 00000000000000000000000000000000 P fef67d497b9646ce8fdcfb3d2727bed4 [sys.catalog]: This master's current role is: LEADER
I20260806 09:33:32.281028 14289 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 09:33:32.286245 14289 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 09:33:32.295202 14289 catalog_manager.cc:1347] Generated new cluster ID: d98399d6012742bf9deee49b81544df6
I20260806 09:33:32.295320 14289 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 09:33:32.322947 14289 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 09:33:32.324246 14289 catalog_manager.cc:1504] Loading token signing keys...
I20260806 09:33:32.334977 14289 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fef67d497b9646ce8fdcfb3d2727bed4: Generated new TSK 0
I20260806 09:33:32.335901 14289 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 09:33:32.348204 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786008495763054-20932-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786008495763054-20932-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786008495763054-20932-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786008495763054-20932-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.113.1:0
--local_ip_for_outbound_sockets=127.20.113.1
--webserver_interface=127.20.113.1
--webserver_port=0
--tserver_master_addrs=127.20.113.62:35037
--builtin_ntp_servers=127.20.113.20:36275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 09:33:32.744379 14300 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:33:32.744678 14300 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:33:32.744846 14300 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:33:32.751984 14300 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:33:32.752069 14300 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:33:32.752267 14300 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.113.1
I20260806 09:33:32.761143 14300 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:36275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786008495763054-20932-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786008495763054-20932-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.113.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786008495763054-20932-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.113.1
--webserver_port=0
--tserver_master_addrs=127.20.113.62:35037
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.113.1
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786008495763054-20932-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:33:32.762390 14300 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:33:32.764127 14300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:33:32.777088 14307 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:33:32.777036 14313 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:33:32.777135 14310 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:33:32.778183 14300 server_base.cc:1034] running on GCE node
I20260806 09:33:32.778755 14300 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:33:32.779690 14300 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:33:32.780936 14300 hybrid_clock.cc:648] HybridClock initialized: now 1786008812780885 us; error 61 us; skew 500 ppm
I20260806 09:33:32.781253 14300 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 09:33:32.783782 14300 webserver.cc:460] Webserver started at http://127.20.113.1:33239/ using document root <none> and password file <none>
I20260806 09:33:32.784545 14300 fs_manager.cc:359] Metadata directory not provided
I20260806 09:33:32.784683 14300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:33:32.785017 14300 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:33:32.787963 14300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786008495763054-20932-0/minicluster-data/ts-0/data/instance:
uuid: "ec8c407e43b34792b91bddd88df2d715"
format_stamp: "Formatted at 2026-08-06 09:33:32 on dist-test-slave-45dx"
server_key: "14318fb67c06867b5e867db2e7136709"
server_key_iv: "ba7ead4c55a37e2836a8f2f4fb07837f"
server_key_version: "encryptionkey@0"
I20260806 09:33:32.788772 14300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786008495763054-20932-0/minicluster-data/ts-0/wal/instance:
uuid: "ec8c407e43b34792b91bddd88df2d715"
format_stamp: "Formatted at 2026-08-06 09:33:32 on dist-test-slave-45dx"
server_key: "14318fb67c06867b5e867db2e7136709"
server_key_iv: "ba7ead4c55a37e2836a8f2f4fb07837f"
server_key_version: "encryptionkey@0"
I20260806 09:33:32.793792 14300 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 09:33:32.797211 14321 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:33:32.798482 14300 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 09:33:32.798660 14300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786008495763054-20932-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786008495763054-20932-0/minicluster-data/ts-0/wal
uuid: "ec8c407e43b34792b91bddd88df2d715"
format_stamp: "Formatted at 2026-08-06 09:33:32 on dist-test-slave-45dx"
server_key: "14318fb67c06867b5e867db2e7136709"
server_key_iv: "ba7ead4c55a37e2836a8f2f4fb07837f"
server_key_version: "encryptionkey@0"
I20260806 09:33:32.798880 14300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786008495763054-20932-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786008495763054-20932-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786008495763054-20932-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:33:32.813685 14300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:33:32.814464 14300 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:33:32.814774 14300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:33:32.815775 14300 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:33:32.817211 14300 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:33:32.817338 14300 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:33:32.817479 14300 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:33:32.817571 14300 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:33:32.851466 14300 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.1:37251
I20260806 09:33:32.851518 14486 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.1:37251 every 8 connection(s)
I20260806 09:33:32.852849 14300 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786008495763054-20932-0/minicluster-data/ts-0/data/info.pb
I20260806 09:33:32.854849 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 14300
I20260806 09:33:32.855058 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786008495763054-20932-0/minicluster-data/ts-0/wal/instance
I20260806 09:33:32.855604 20932 external_mini_cluster.cc:1428] Setting key 3e1ba59c562cac5174ac5798cd394d23
I20260806 09:33:32.859416 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786008495763054-20932-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786008495763054-20932-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786008495763054-20932-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786008495763054-20932-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.113.2:0
--local_ip_for_outbound_sockets=127.20.113.2
--webserver_interface=127.20.113.2
--webserver_port=0
--tserver_master_addrs=127.20.113.62:35037
--builtin_ntp_servers=127.20.113.20:36275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 09:33:32.872836 14491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:32.855522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.1:39007 (local address 127.20.113.62:35037)
0806 09:33:32.864692 (+ 9170us) server_negotiation.cc:206] Beginning negotiation
0806 09:33:32.864697 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:33:32.864713 (+ 16us) server_negotiation.cc:407] Connection header received
0806 09:33:32.864760 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:33:32.864764 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:33:32.864829 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:33:32.864922 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:33:32.866320 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:32.866999 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:32.869902 (+ 2903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:32.870278 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:32.870387 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:32.872008 (+ 1621us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:33:32.872012 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:33:32.872015 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:33:32.872031 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:33:32.872086 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:33:32.872098 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:33:32.872168 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 09:33:32.872504 (+ 336us) server_negotiation.cc:299] Negotiation successful
0806 09:33:32.872580 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9057,"thread_start_us":8958,"threads_started":1}
I20260806 09:33:32.874398 14493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:32.856023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:35037 (local address 127.20.113.1:39007)
0806 09:33:32.857473 (+ 1450us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:32.857518 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 09:33:32.859773 (+ 2255us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:33:32.865198 (+ 5425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:33:32.865211 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:33:32.865327 (+ 116us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:33:32.866144 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:33:32.866171 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:33:32.867160 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:33:32.867168 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:33:32.869670 (+ 2502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:33:32.869690 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:33:32.870518 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:33:32.870528 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:33:32.870877 (+ 349us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:32.871657 (+ 780us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 09:33:32.871663 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:33:32.871717 (+ 54us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 09:33:32.871817 (+ 100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:33:32.872207 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:33:32.872223 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:33:32.872235 (+ 12us) client_negotiation.cc:769] Sending connection context
0806 09:33:32.872493 (+ 258us) client_negotiation.cc:240] Negotiation successful
0806 09:33:32.872918 (+ 425us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":935,"thread_start_us":754,"threads_started":1}
I20260806 09:33:32.876092 14488 heartbeater.cc:347] Connected to a master server at 127.20.113.62:35037
I20260806 09:33:32.876480 14488 heartbeater.cc:464] Registering TS with master...
I20260806 09:33:32.877436 14488 heartbeater.cc:511] Master 127.20.113.62:35037 requested a full tablet report, sending...
I20260806 09:33:32.879592 14225 ts_manager.cc:194] Registered new tserver with Master: ec8c407e43b34792b91bddd88df2d715 (127.20.113.1:37251)
I20260806 09:33:32.881147 14225 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.113.1:39007
W20260806 09:33:33.212684 14495 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:33:33.213011 14495 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:33:33.213096 14495 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:33:33.220084 14495 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:33:33.220187 14495 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:33:33.220288 14495 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.113.2
I20260806 09:33:33.229348 14495 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:36275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786008495763054-20932-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786008495763054-20932-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.113.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786008495763054-20932-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.113.2
--webserver_port=0
--tserver_master_addrs=127.20.113.62:35037
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.113.2
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786008495763054-20932-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:33:33.230593 14495 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:33:33.233080 14495 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:33:33.248150 14502 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:33:33.248122 14503 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:33:33.248749 14505 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:33:33.249238 14495 server_base.cc:1034] running on GCE node
I20260806 09:33:33.249864 14495 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:33:33.250859 14495 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:33:33.252125 14495 hybrid_clock.cc:648] HybridClock initialized: now 1786008813252054 us; error 53 us; skew 500 ppm
I20260806 09:33:33.252535 14495 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 09:33:33.254874 14495 webserver.cc:460] Webserver started at http://127.20.113.2:45579/ using document root <none> and password file <none>
I20260806 09:33:33.255563 14495 fs_manager.cc:359] Metadata directory not provided
I20260806 09:33:33.255704 14495 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:33:33.256038 14495 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:33:33.259040 14495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786008495763054-20932-0/minicluster-data/ts-1/data/instance:
uuid: "f645147ff369424ea3ef7872db16381f"
format_stamp: "Formatted at 2026-08-06 09:33:33 on dist-test-slave-45dx"
server_key: "613accf8b47a414499cad227f7af39dc"
server_key_iv: "b0e5a73c30280c2438d162b7628951c0"
server_key_version: "encryptionkey@0"
I20260806 09:33:33.259827 14495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786008495763054-20932-0/minicluster-data/ts-1/wal/instance:
uuid: "f645147ff369424ea3ef7872db16381f"
format_stamp: "Formatted at 2026-08-06 09:33:33 on dist-test-slave-45dx"
server_key: "613accf8b47a414499cad227f7af39dc"
server_key_iv: "b0e5a73c30280c2438d162b7628951c0"
server_key_version: "encryptionkey@0"
I20260806 09:33:33.265051 14495 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260806 09:33:33.268487 14514 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:33:33.269812 14495 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 09:33:33.270017 14495 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786008495763054-20932-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786008495763054-20932-0/minicluster-data/ts-1/wal
uuid: "f645147ff369424ea3ef7872db16381f"
format_stamp: "Formatted at 2026-08-06 09:33:33 on dist-test-slave-45dx"
server_key: "613accf8b47a414499cad227f7af39dc"
server_key_iv: "b0e5a73c30280c2438d162b7628951c0"
server_key_version: "encryptionkey@0"
I20260806 09:33:33.270241 14495 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786008495763054-20932-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786008495763054-20932-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786008495763054-20932-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:33:33.290059 14495 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:33:33.290825 14495 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:33:33.291126 14495 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:33:33.292054 14495 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:33:33.293645 14495 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:33:33.293753 14495 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:33:33.293876 14495 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:33:33.293965 14495 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:33:33.327576 14495 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.2:41673
I20260806 09:33:33.327668 14669 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.2:41673 every 8 connection(s)
I20260806 09:33:33.328994 14495 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786008495763054-20932-0/minicluster-data/ts-1/data/info.pb
I20260806 09:33:33.338704 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 14495
I20260806 09:33:33.338915 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786008495763054-20932-0/minicluster-data/ts-1/wal/instance
I20260806 09:33:33.339375 20932 external_mini_cluster.cc:1428] Setting key 4b10e6d29e506b6eb3e0f80ddd8513f6
I20260806 09:33:33.342432 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786008495763054-20932-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786008495763054-20932-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786008495763054-20932-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786008495763054-20932-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.113.3:0
--local_ip_for_outbound_sockets=127.20.113.3
--webserver_interface=127.20.113.3
--webserver_port=0
--tserver_master_addrs=127.20.113.62:35037
--builtin_ntp_servers=127.20.113.20:36275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 09:33:33.342468 14491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:33.331651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.2:38797 (local address 127.20.113.62:35037)
0806 09:33:33.331842 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0806 09:33:33.331846 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 09:33:33.333278 (+ 1432us) server_negotiation.cc:407] Connection header received
0806 09:33:33.335674 (+ 2396us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:33:33.335679 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:33:33.335777 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:33:33.335895 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:33:33.337085 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:33.337820 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:33.339501 (+ 1681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:33.339913 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:33.340053 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:33.341601 (+ 1548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:33:33.341606 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:33:33.341610 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:33:33.341627 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:33:33.341659 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:33:33.341670 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:33:33.341781 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 09:33:33.342031 (+ 250us) server_negotiation.cc:299] Negotiation successful
0806 09:33:33.342127 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 09:33:33.343468 14674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:33.332065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:35037 (local address 127.20.113.2:38797)
0806 09:33:33.333100 (+ 1035us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:33.333143 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 09:33:33.335391 (+ 2248us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:33:33.336149 (+ 758us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:33:33.336157 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:33:33.336250 (+ 93us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:33:33.336910 (+ 660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:33:33.336928 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:33:33.337991 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:33:33.338003 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:33:33.339354 (+ 1351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:33:33.339368 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:33:33.340148 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:33:33.340157 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:33:33.340484 (+ 327us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:33.341261 (+ 777us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 09:33:33.341264 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:33:33.341309 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 09:33:33.341399 (+ 90us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:33:33.341798 (+ 399us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:33:33.341811 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:33:33.341821 (+ 10us) client_negotiation.cc:769] Sending connection context
0806 09:33:33.342001 (+ 180us) client_negotiation.cc:240] Negotiation successful
0806 09:33:33.342412 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":577,"thread_start_us":369,"threads_started":1}
I20260806 09:33:33.344852 14671 heartbeater.cc:347] Connected to a master server at 127.20.113.62:35037
I20260806 09:33:33.345197 14671 heartbeater.cc:464] Registering TS with master...
I20260806 09:33:33.346115 14671 heartbeater.cc:511] Master 127.20.113.62:35037 requested a full tablet report, sending...
I20260806 09:33:33.347698 14225 ts_manager.cc:194] Registered new tserver with Master: f645147ff369424ea3ef7872db16381f (127.20.113.2:41673)
I20260806 09:33:33.348573 14225 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.113.2:38797
W20260806 09:33:33.699198 14676 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:33:33.699465 14676 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:33:33.699553 14676 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:33:33.706835 14676 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:33:33.706923 14676 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:33:33.707026 14676 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.113.3
I20260806 09:33:33.715726 14676 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:36275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786008495763054-20932-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786008495763054-20932-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.113.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786008495763054-20932-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.113.3
--webserver_port=0
--tserver_master_addrs=127.20.113.62:35037
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.113.3
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786008495763054-20932-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:33:33.716894 14676 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:33:33.718587 14676 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:33:33.732177 14685 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:33:33.732281 14686 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:33:33.732360 14690 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:33:33.733337 14676 server_base.cc:1034] running on GCE node
I20260806 09:33:33.733984 14676 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:33:33.734851 14676 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:33:33.736130 14676 hybrid_clock.cc:648] HybridClock initialized: now 1786008813736073 us; error 37 us; skew 500 ppm
I20260806 09:33:33.736515 14676 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 09:33:33.738757 14676 webserver.cc:460] Webserver started at http://127.20.113.3:38923/ using document root <none> and password file <none>
I20260806 09:33:33.739456 14676 fs_manager.cc:359] Metadata directory not provided
I20260806 09:33:33.739615 14676 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:33:33.739943 14676 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:33:33.742862 14676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786008495763054-20932-0/minicluster-data/ts-2/data/instance:
uuid: "e996f95d6d8a440a8d5d122cd9c2cf95"
format_stamp: "Formatted at 2026-08-06 09:33:33 on dist-test-slave-45dx"
server_key: "90a731648092d82b59351fdde172307e"
server_key_iv: "ee0d9c9a3c2d1b3ab6a784e5f5a0f9c6"
server_key_version: "encryptionkey@0"
I20260806 09:33:33.743711 14676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786008495763054-20932-0/minicluster-data/ts-2/wal/instance:
uuid: "e996f95d6d8a440a8d5d122cd9c2cf95"
format_stamp: "Formatted at 2026-08-06 09:33:33 on dist-test-slave-45dx"
server_key: "90a731648092d82b59351fdde172307e"
server_key_iv: "ee0d9c9a3c2d1b3ab6a784e5f5a0f9c6"
server_key_version: "encryptionkey@0"
I20260806 09:33:33.748916 14676 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260806 09:33:33.752262 14698 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:33:33.753638 14676 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 09:33:33.753819 14676 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786008495763054-20932-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786008495763054-20932-0/minicluster-data/ts-2/wal
uuid: "e996f95d6d8a440a8d5d122cd9c2cf95"
format_stamp: "Formatted at 2026-08-06 09:33:33 on dist-test-slave-45dx"
server_key: "90a731648092d82b59351fdde172307e"
server_key_iv: "ee0d9c9a3c2d1b3ab6a784e5f5a0f9c6"
server_key_version: "encryptionkey@0"
I20260806 09:33:33.754050 14676 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786008495763054-20932-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786008495763054-20932-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786008495763054-20932-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:33:33.774143 14676 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:33:33.774909 14676 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:33:33.775218 14676 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:33:33.776201 14676 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:33:33.777789 14676 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:33:33.777926 14676 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:33:33.778056 14676 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:33:33.778148 14676 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:33:33.813006 14676 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.3:44583
I20260806 09:33:33.813102 14852 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.3:44583 every 8 connection(s)
I20260806 09:33:33.814415 14676 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786008495763054-20932-0/minicluster-data/ts-2/data/info.pb
I20260806 09:33:33.818053 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 14676
I20260806 09:33:33.818240 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786008495763054-20932-0/minicluster-data/ts-2/wal/instance
I20260806 09:33:33.818739 20932 external_mini_cluster.cc:1428] Setting key ba8d1b4eaab8f201731f35f7cb581a54
I20260806 09:33:33.828100 14491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:33.817484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.3:47803 (local address 127.20.113.62:35037)
0806 09:33:33.817743 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0806 09:33:33.817751 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 09:33:33.819454 (+ 1703us) server_negotiation.cc:407] Connection header received
0806 09:33:33.821710 (+ 2256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:33:33.821717 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:33:33.821820 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:33:33.821944 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:33:33.823023 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:33.823788 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:33.825415 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:33.825821 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:33.825961 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:33.827261 (+ 1300us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:33:33.827265 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:33:33.827269 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:33:33.827288 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:33:33.827320 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:33:33.827332 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:33:33.827431 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 09:33:33.827662 (+ 231us) server_negotiation.cc:299] Negotiation successful
0806 09:33:33.827761 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260806 09:33:33.829162 14856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:33.818060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:35037 (local address 127.20.113.3:47803)
0806 09:33:33.819276 (+ 1216us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:33.819334 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0806 09:33:33.821348 (+ 2014us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:33:33.822184 (+ 836us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:33:33.822192 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:33:33.822279 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:33:33.822853 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:33:33.822870 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:33:33.823945 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:33:33.823952 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:33:33.825259 (+ 1307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:33:33.825272 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:33:33.826053 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:33:33.826061 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:33:33.826290 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:33.826938 (+ 648us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 09:33:33.826941 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:33:33.826978 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 09:33:33.827055 (+ 77us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:33:33.827442 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:33:33.827456 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:33:33.827470 (+ 14us) client_negotiation.cc:769] Sending connection context
0806 09:33:33.827647 (+ 177us) client_negotiation.cc:240] Negotiation successful
0806 09:33:33.828064 (+ 417us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":668,"thread_start_us":421,"threads_started":1}
I20260806 09:33:33.830426 14854 heartbeater.cc:347] Connected to a master server at 127.20.113.62:35037
I20260806 09:33:33.830721 14854 heartbeater.cc:464] Registering TS with master...
I20260806 09:33:33.831480 14854 heartbeater.cc:511] Master 127.20.113.62:35037 requested a full tablet report, sending...
I20260806 09:33:33.832902 14225 ts_manager.cc:194] Registered new tserver with Master: e996f95d6d8a440a8d5d122cd9c2cf95 (127.20.113.3:44583)
I20260806 09:33:33.833938 14225 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.113.3:47803
I20260806 09:33:33.834045 20932 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 09:33:33.844089 14491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:33.838677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57542 (local address 127.20.113.62:35037)
0806 09:33:33.838959 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0806 09:33:33.838965 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:33:33.839283 (+ 318us) server_negotiation.cc:407] Connection header received
0806 09:33:33.839426 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:33:33.839430 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:33:33.839483 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:33:33.839561 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:33:33.839993 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:33.840664 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:33.841957 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:33.842345 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:33.842436 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:33.843475 (+ 1039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:33:33.843479 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:33:33.843483 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:33:33.843498 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:33:33.843526 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:33:33.843536 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:33:33.843603 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 09:33:33.843719 (+ 116us) server_negotiation.cc:299] Negotiation successful
0806 09:33:33.843783 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 09:33:33.847668 14225 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57542:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 09:33:33.854530 14225 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 09:33:33.880501 14872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:33.867016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.3:44583 (local address 127.0.0.1:49202)
0806 09:33:33.868569 (+ 1553us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:33.868619 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 09:33:33.868870 (+ 251us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:33:33.870875 (+ 2005us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:33:33.870888 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:33:33.870931 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:33:33.871474 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:33:33.871501 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:33:33.873997 (+ 2496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:33:33.874006 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:33:33.876803 (+ 2797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:33:33.876818 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:33:33.879406 (+ 2588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:33:33.879415 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:33:33.879710 (+ 295us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:33.879786 (+ 76us) client_negotiation.cc:769] Sending connection context
0806 09:33:33.879918 (+ 132us) client_negotiation.cc:240] Negotiation successful
0806 09:33:33.880127 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1223,"thread_start_us":364,"threads_started":1}
I20260806 09:33:33.882090 14874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:33.867358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49202 (local address 127.20.113.3:44583)
0806 09:33:33.870125 (+ 2767us) server_negotiation.cc:206] Beginning negotiation
0806 09:33:33.870134 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 09:33:33.870172 (+ 38us) server_negotiation.cc:407] Connection header received
0806 09:33:33.870274 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:33:33.870281 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:33:33.870558 (+ 277us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:33:33.871727 (+ 1169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:33:33.872088 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:33.873793 (+ 1705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:33.877164 (+ 3371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:33.879135 (+ 1971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:33.881053 (+ 1918us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:33.881351 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0806 09:33:33.881489 (+ 138us) server_negotiation.cc:299] Negotiation successful
0806 09:33:33.881679 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2593,"thread_start_us":2418,"threads_started":1}
I20260806 09:33:33.885018 14488 heartbeater.cc:503] Master 127.20.113.62:35037 was elected leader, sending a full tablet report...
I20260806 09:33:33.886385 14764 tablet_service.cc:1467] Processing CreateTablet for tablet 6782134434a140adae3bcd5f43e6b2d5 (DEFAULT_TABLE table=test-table [id=3a156e706cd54692a896ccb0bfc66b12]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:33:33.888047 14764 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6782134434a140adae3bcd5f43e6b2d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:33:33.888860 14873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:33.867334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.2:41673 (local address 127.0.0.1:34964)
0806 09:33:33.869395 (+ 2061us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:33.869412 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 09:33:33.869535 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:33:33.882187 (+ 12652us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:33:33.882192 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:33:33.882219 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:33:33.882609 (+ 390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:33:33.882626 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:33:33.884464 (+ 1838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:33:33.884471 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:33:33.886647 (+ 2176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:33:33.886661 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:33:33.888072 (+ 1411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:33:33.888077 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:33:33.888238 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:33.888272 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 09:33:33.888381 (+ 109us) client_negotiation.cc:240] Negotiation successful
0806 09:33:33.888501 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1907,"mutex_wait_us":54,"thread_start_us":2265,"threads_started":1}
I20260806 09:33:33.888888 14878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:33.867933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34964 (local address 127.20.113.2:41673)
0806 09:33:33.881567 (+ 13634us) server_negotiation.cc:206] Beginning negotiation
0806 09:33:33.881578 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 09:33:33.881608 (+ 30us) server_negotiation.cc:407] Connection header received
0806 09:33:33.881698 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:33:33.881705 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:33:33.881966 (+ 261us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:33:33.882173 (+ 207us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:33:33.882840 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:33.884347 (+ 1507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:33.886873 (+ 2526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:33.887877 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:33.888025 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:33.888265 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0806 09:33:33.888422 (+ 157us) server_negotiation.cc:299] Negotiation successful
0806 09:33:33.888578 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13438,"thread_start_us":13322,"threads_started":1}
I20260806 09:33:33.890552 14880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:33.869397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.1:37251 (local address 127.0.0.1:55126)
0806 09:33:33.880213 (+ 10816us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:33.880236 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 09:33:33.880395 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:33:33.882463 (+ 2068us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:33:33.882468 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:33:33.882499 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:33:33.882999 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:33:33.883012 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:33:33.884376 (+ 1364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:33:33.884384 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:33:33.887838 (+ 3454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:33:33.887865 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:33:33.889731 (+ 1866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:33:33.889738 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:33:33.889923 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:33.889955 (+ 32us) client_negotiation.cc:769] Sending connection context
0806 09:33:33.890042 (+ 87us) client_negotiation.cc:240] Negotiation successful
0806 09:33:33.890170 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10650,"mutex_wait_us":69,"thread_start_us":10455,"threads_started":1}
I20260806 09:33:33.890654 14879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:33.868697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55126 (local address 127.20.113.1:37251)
0806 09:33:33.881655 (+ 12958us) server_negotiation.cc:206] Beginning negotiation
0806 09:33:33.881661 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:33:33.881686 (+ 25us) server_negotiation.cc:407] Connection header received
0806 09:33:33.881787 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:33:33.881792 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:33:33.881948 (+ 156us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:33:33.882208 (+ 260us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:33:33.883175 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:33.884193 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:33.888086 (+ 3893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:33.889537 (+ 1451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:33.889684 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:33.889948 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0806 09:33:33.890122 (+ 174us) server_negotiation.cc:299] Negotiation successful
0806 09:33:33.890282 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12604,"thread_start_us":12719,"threads_started":1}
I20260806 09:33:33.892892 14579 tablet_service.cc:1467] Processing CreateTablet for tablet 6782134434a140adae3bcd5f43e6b2d5 (DEFAULT_TABLE table=test-table [id=3a156e706cd54692a896ccb0bfc66b12]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:33:33.894166 14579 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6782134434a140adae3bcd5f43e6b2d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:33:33.894371 14389 tablet_service.cc:1467] Processing CreateTablet for tablet 6782134434a140adae3bcd5f43e6b2d5 (DEFAULT_TABLE table=test-table [id=3a156e706cd54692a896ccb0bfc66b12]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:33:33.895711 14389 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6782134434a140adae3bcd5f43e6b2d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:33:33.905866 14882 tablet_bootstrap.cc:492] T 6782134434a140adae3bcd5f43e6b2d5 P f645147ff369424ea3ef7872db16381f: Bootstrap starting.
I20260806 09:33:33.908667 14881 tablet_bootstrap.cc:492] T 6782134434a140adae3bcd5f43e6b2d5 P e996f95d6d8a440a8d5d122cd9c2cf95: Bootstrap starting.
I20260806 09:33:33.910151 14882 tablet_bootstrap.cc:654] T 6782134434a140adae3bcd5f43e6b2d5 P f645147ff369424ea3ef7872db16381f: Neither blocks nor log segments found. Creating new log.
I20260806 09:33:33.911239 14883 tablet_bootstrap.cc:492] T 6782134434a140adae3bcd5f43e6b2d5 P ec8c407e43b34792b91bddd88df2d715: Bootstrap starting.
I20260806 09:33:33.911835 14882 log.cc:824] T 6782134434a140adae3bcd5f43e6b2d5 P f645147ff369424ea3ef7872db16381f: Log is configured to *not* fsync() on all Append() calls
I20260806 09:33:33.912276 14881 tablet_bootstrap.cc:654] T 6782134434a140adae3bcd5f43e6b2d5 P e996f95d6d8a440a8d5d122cd9c2cf95: Neither blocks nor log segments found. Creating new log.
I20260806 09:33:33.913796 14881 log.cc:824] T 6782134434a140adae3bcd5f43e6b2d5 P e996f95d6d8a440a8d5d122cd9c2cf95: Log is configured to *not* fsync() on all Append() calls
I20260806 09:33:33.914029 14883 tablet_bootstrap.cc:654] T 6782134434a140adae3bcd5f43e6b2d5 P ec8c407e43b34792b91bddd88df2d715: Neither blocks nor log segments found. Creating new log.
I20260806 09:33:33.915277 14883 log.cc:824] T 6782134434a140adae3bcd5f43e6b2d5 P ec8c407e43b34792b91bddd88df2d715: Log is configured to *not* fsync() on all Append() calls
I20260806 09:33:33.916834 14882 tablet_bootstrap.cc:492] T 6782134434a140adae3bcd5f43e6b2d5 P f645147ff369424ea3ef7872db16381f: No bootstrap required, opened a new log
I20260806 09:33:33.916900 14881 tablet_bootstrap.cc:492] T 6782134434a140adae3bcd5f43e6b2d5 P e996f95d6d8a440a8d5d122cd9c2cf95: No bootstrap required, opened a new log
I20260806 09:33:33.917131 14882 ts_tablet_manager.cc:1397] T 6782134434a140adae3bcd5f43e6b2d5 P f645147ff369424ea3ef7872db16381f: Time spent bootstrapping tablet: real 0.011s user 0.009s sys 0.000s
I20260806 09:33:33.917207 14881 ts_tablet_manager.cc:1397] T 6782134434a140adae3bcd5f43e6b2d5 P e996f95d6d8a440a8d5d122cd9c2cf95: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260806 09:33:33.918363 14883 tablet_bootstrap.cc:492] T 6782134434a140adae3bcd5f43e6b2d5 P ec8c407e43b34792b91bddd88df2d715: No bootstrap required, opened a new log
I20260806 09:33:33.918707 14883 ts_tablet_manager.cc:1397] T 6782134434a140adae3bcd5f43e6b2d5 P ec8c407e43b34792b91bddd88df2d715: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260806 09:33:33.925472 14882 raft_consensus.cc:348] T 6782134434a140adae3bcd5f43e6b2d5 P f645147ff369424ea3ef7872db16381f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e996f95d6d8a440a8d5d122cd9c2cf95" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 44583 } } peers { permanent_uuid: "f645147ff369424ea3ef7872db16381f" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 41673 } } peers { permanent_uuid: "ec8c407e43b34792b91bddd88df2d715" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 37251 } }
I20260806 09:33:33.926151 14882 raft_consensus.cc:374] T 6782134434a140adae3bcd5f43e6b2d5 P f645147ff369424ea3ef7872db16381f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:33:33.926311 14882 raft_consensus.cc:729] T 6782134434a140adae3bcd5f43e6b2d5 P f645147ff369424ea3ef7872db16381f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f645147ff369424ea3ef7872db16381f, State: Initialized, Role: FOLLOWER
I20260806 09:33:33.927002 14882 consensus_queue.cc:260] T 6782134434a140adae3bcd5f43e6b2d5 P f645147ff369424ea3ef7872db16381f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e996f95d6d8a440a8d5d122cd9c2cf95" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 44583 } } peers { permanent_uuid: "f645147ff369424ea3ef7872db16381f" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 41673 } } peers { permanent_uuid: "ec8c407e43b34792b91bddd88df2d715" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 37251 } }
I20260806 09:33:33.928180 14671 heartbeater.cc:503] Master 127.20.113.62:35037 was elected leader, sending a full tablet report...
I20260806 09:33:33.928144 14881 raft_consensus.cc:348] T 6782134434a140adae3bcd5f43e6b2d5 P e996f95d6d8a440a8d5d122cd9c2cf95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e996f95d6d8a440a8d5d122cd9c2cf95" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 44583 } } peers { permanent_uuid: "f645147ff369424ea3ef7872db16381f" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 41673 } } peers { permanent_uuid: "ec8c407e43b34792b91bddd88df2d715" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 37251 } }
I20260806 09:33:33.928622 14881 raft_consensus.cc:374] T 6782134434a140adae3bcd5f43e6b2d5 P e996f95d6d8a440a8d5d122cd9c2cf95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:33:33.928716 14881 raft_consensus.cc:729] T 6782134434a140adae3bcd5f43e6b2d5 P e996f95d6d8a440a8d5d122cd9c2cf95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e996f95d6d8a440a8d5d122cd9c2cf95, State: Initialized, Role: FOLLOWER
I20260806 09:33:33.929013 14882 ts_tablet_manager.cc:1428] T 6782134434a140adae3bcd5f43e6b2d5 P f645147ff369424ea3ef7872db16381f: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
I20260806 09:33:33.929184 14883 raft_consensus.cc:348] T 6782134434a140adae3bcd5f43e6b2d5 P ec8c407e43b34792b91bddd88df2d715 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e996f95d6d8a440a8d5d122cd9c2cf95" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 44583 } } peers { permanent_uuid: "f645147ff369424ea3ef7872db16381f" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 41673 } } peers { permanent_uuid: "ec8c407e43b34792b91bddd88df2d715" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 37251 } }
I20260806 09:33:33.929353 14881 consensus_queue.cc:260] T 6782134434a140adae3bcd5f43e6b2d5 P e996f95d6d8a440a8d5d122cd9c2cf95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e996f95d6d8a440a8d5d122cd9c2cf95" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 44583 } } peers { permanent_uuid: "f645147ff369424ea3ef7872db16381f" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 41673 } } peers { permanent_uuid: "ec8c407e43b34792b91bddd88df2d715" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 37251 } }
I20260806 09:33:33.929605 14883 raft_consensus.cc:374] T 6782134434a140adae3bcd5f43e6b2d5 P ec8c407e43b34792b91bddd88df2d715 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:33:33.929730 14883 raft_consensus.cc:729] T 6782134434a140adae3bcd5f43e6b2d5 P ec8c407e43b34792b91bddd88df2d715 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec8c407e43b34792b91bddd88df2d715, State: Initialized, Role: FOLLOWER
I20260806 09:33:33.930518 14883 consensus_queue.cc:260] T 6782134434a140adae3bcd5f43e6b2d5 P ec8c407e43b34792b91bddd88df2d715 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e996f95d6d8a440a8d5d122cd9c2cf95" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 44583 } } peers { permanent_uuid: "f645147ff369424ea3ef7872db16381f" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 41673 } } peers { permanent_uuid: "ec8c407e43b34792b91bddd88df2d715" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 37251 } }
I20260806 09:33:33.931272 14881 ts_tablet_manager.cc:1428] T 6782134434a140adae3bcd5f43e6b2d5 P e996f95d6d8a440a8d5d122cd9c2cf95: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260806 09:33:33.932569 14854 heartbeater.cc:503] Master 127.20.113.62:35037 was elected leader, sending a full tablet report...
I20260806 09:33:33.932669 14883 ts_tablet_manager.cc:1428] T 6782134434a140adae3bcd5f43e6b2d5 P ec8c407e43b34792b91bddd88df2d715: Time spent starting tablet: real 0.014s user 0.004s sys 0.010s
I20260806 09:33:33.994599 14891 raft_consensus.cc:482] T 6782134434a140adae3bcd5f43e6b2d5 P f645147ff369424ea3ef7872db16381f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 09:33:33.995004 14891 raft_consensus.cc:504] T 6782134434a140adae3bcd5f43e6b2d5 P f645147ff369424ea3ef7872db16381f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e996f95d6d8a440a8d5d122cd9c2cf95" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 44583 } } peers { permanent_uuid: "f645147ff369424ea3ef7872db16381f" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 41673 } } peers { permanent_uuid: "ec8c407e43b34792b91bddd88df2d715" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 37251 } }
I20260806 09:33:33.996840 14891 leader_election.cc:288] T 6782134434a140adae3bcd5f43e6b2d5 P f645147ff369424ea3ef7872db16381f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e996f95d6d8a440a8d5d122cd9c2cf95 (127.20.113.3:44583), ec8c407e43b34792b91bddd88df2d715 (127.20.113.1:37251)
I20260806 09:33:34.004578 14874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:33.997224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.2:55821 (local address 127.20.113.3:44583)
0806 09:33:33.997426 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0806 09:33:33.997432 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:33:33.998395 (+ 963us) server_negotiation.cc:407] Connection header received
0806 09:33:33.998613 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:33:33.998621 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:33:33.998715 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:33:33.998782 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:33:33.999592 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:34.000450 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:34.002600 (+ 2150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:34.003522 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:34.003706 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:34.003792 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 09:33:34.004186 (+ 394us) server_negotiation.cc:299] Negotiation successful
0806 09:33:34.004312 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260806 09:33:34.004583 14879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:33.997391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.2:58299 (local address 127.20.113.1:37251)
0806 09:33:33.997607 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0806 09:33:33.997615 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 09:33:33.998499 (+ 884us) server_negotiation.cc:407] Connection header received
0806 09:33:33.998718 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:33:33.998722 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:33:33.998831 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:33:33.998927 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:33:33.999746 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:34.000562 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:34.002678 (+ 2116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:34.003487 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:34.003647 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:34.003726 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 09:33:34.004180 (+ 454us) server_negotiation.cc:299] Negotiation successful
0806 09:33:34.004306 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 09:33:34.004726 14895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:33.997427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.1:37251 (local address 127.20.113.2:58299)
0806 09:33:33.998361 (+ 934us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:33.998385 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 09:33:33.998567 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:33:33.998969 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:33:33.998972 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:33:33.999023 (+ 51us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:33:33.999566 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:33:33.999578 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:33:34.000712 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:33:34.000715 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:33:34.002407 (+ 1692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:33:34.002420 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:33:34.003786 (+ 1366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:33:34.003795 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:33:34.003961 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:34.004011 (+ 50us) client_negotiation.cc:769] Sending connection context
0806 09:33:34.004131 (+ 120us) client_negotiation.cc:240] Negotiation successful
0806 09:33:34.004274 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":810,"mutex_wait_us":56,"spinlock_wait_cycles":3328,"thread_start_us":543,"threads_started":1}
I20260806 09:33:34.004756 14894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:33.997205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.3:44583 (local address 127.20.113.2:55821)
0806 09:33:33.998247 (+ 1042us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:33.998266 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 09:33:33.998408 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:33:33.998861 (+ 453us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:33:33.998869 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:33:33.998902 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:33:33.999305 (+ 403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:33:33.999320 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:33:34.000587 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:33:34.000594 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:33:34.002382 (+ 1788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:33:34.002395 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:33:34.003837 (+ 1442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:33:34.003844 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:33:34.003971 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:34.004015 (+ 44us) client_negotiation.cc:769] Sending connection context
0806 09:33:34.004135 (+ 120us) client_negotiation.cc:240] Negotiation successful
0806 09:33:34.004284 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":873,"mutex_wait_us":77,"thread_start_us":632,"threads_started":1}
I20260806 09:33:34.005896 14418 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6782134434a140adae3bcd5f43e6b2d5" candidate_uuid: "f645147ff369424ea3ef7872db16381f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec8c407e43b34792b91bddd88df2d715" is_pre_election: true
I20260806 09:33:34.005924 14796 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6782134434a140adae3bcd5f43e6b2d5" candidate_uuid: "f645147ff369424ea3ef7872db16381f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e996f95d6d8a440a8d5d122cd9c2cf95" is_pre_election: true
I20260806 09:33:34.006438 14418 raft_consensus.cc:2454] T 6782134434a140adae3bcd5f43e6b2d5 P ec8c407e43b34792b91bddd88df2d715 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f645147ff369424ea3ef7872db16381f in term 0.
I20260806 09:33:34.006444 14796 raft_consensus.cc:2454] T 6782134434a140adae3bcd5f43e6b2d5 P e996f95d6d8a440a8d5d122cd9c2cf95 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f645147ff369424ea3ef7872db16381f in term 0.
I20260806 09:33:34.007157 14517 leader_election.cc:302] T 6782134434a140adae3bcd5f43e6b2d5 P f645147ff369424ea3ef7872db16381f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: e996f95d6d8a440a8d5d122cd9c2cf95, ec8c407e43b34792b91bddd88df2d715, f645147ff369424ea3ef7872db16381f; no voters:
I20260806 09:33:34.007539 14891 raft_consensus.cc:2781] T 6782134434a140adae3bcd5f43e6b2d5 P f645147ff369424ea3ef7872db16381f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 09:33:34.007704 14891 raft_consensus.cc:482] T 6782134434a140adae3bcd5f43e6b2d5 P f645147ff369424ea3ef7872db16381f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 09:33:34.007828 14891 raft_consensus.cc:3037] T 6782134434a140adae3bcd5f43e6b2d5 P f645147ff369424ea3ef7872db16381f [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:33:34.010181 14891 raft_consensus.cc:504] T 6782134434a140adae3bcd5f43e6b2d5 P f645147ff369424ea3ef7872db16381f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e996f95d6d8a440a8d5d122cd9c2cf95" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 44583 } } peers { permanent_uuid: "f645147ff369424ea3ef7872db16381f" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 41673 } } peers { permanent_uuid: "ec8c407e43b34792b91bddd88df2d715" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 37251 } }
I20260806 09:33:34.010826 14891 leader_election.cc:288] T 6782134434a140adae3bcd5f43e6b2d5 P f645147ff369424ea3ef7872db16381f [CANDIDATE]: Term 1 election: Requested vote from peers e996f95d6d8a440a8d5d122cd9c2cf95 (127.20.113.3:44583), ec8c407e43b34792b91bddd88df2d715 (127.20.113.1:37251)
I20260806 09:33:34.011304 14796 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6782134434a140adae3bcd5f43e6b2d5" candidate_uuid: "f645147ff369424ea3ef7872db16381f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e996f95d6d8a440a8d5d122cd9c2cf95"
I20260806 09:33:34.011351 14418 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6782134434a140adae3bcd5f43e6b2d5" candidate_uuid: "f645147ff369424ea3ef7872db16381f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec8c407e43b34792b91bddd88df2d715"
I20260806 09:33:34.011534 14796 raft_consensus.cc:3037] T 6782134434a140adae3bcd5f43e6b2d5 P e996f95d6d8a440a8d5d122cd9c2cf95 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:33:34.011543 14418 raft_consensus.cc:3037] T 6782134434a140adae3bcd5f43e6b2d5 P ec8c407e43b34792b91bddd88df2d715 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:33:34.013767 14796 raft_consensus.cc:2454] T 6782134434a140adae3bcd5f43e6b2d5 P e996f95d6d8a440a8d5d122cd9c2cf95 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f645147ff369424ea3ef7872db16381f in term 1.
I20260806 09:33:34.013877 14418 raft_consensus.cc:2454] T 6782134434a140adae3bcd5f43e6b2d5 P ec8c407e43b34792b91bddd88df2d715 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f645147ff369424ea3ef7872db16381f in term 1.
I20260806 09:33:34.014199 14517 leader_election.cc:302] T 6782134434a140adae3bcd5f43e6b2d5 P f645147ff369424ea3ef7872db16381f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e996f95d6d8a440a8d5d122cd9c2cf95, f645147ff369424ea3ef7872db16381f; no voters:
I20260806 09:33:34.014544 14891 raft_consensus.cc:2781] T 6782134434a140adae3bcd5f43e6b2d5 P f645147ff369424ea3ef7872db16381f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:33:34.015035 14891 raft_consensus.cc:686] T 6782134434a140adae3bcd5f43e6b2d5 P f645147ff369424ea3ef7872db16381f [term 1 LEADER]: Becoming Leader. State: Replica: f645147ff369424ea3ef7872db16381f, State: Running, Role: LEADER
I20260806 09:33:34.015563 14891 consensus_queue.cc:237] T 6782134434a140adae3bcd5f43e6b2d5 P f645147ff369424ea3ef7872db16381f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e996f95d6d8a440a8d5d122cd9c2cf95" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 44583 } } peers { permanent_uuid: "f645147ff369424ea3ef7872db16381f" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 41673 } } peers { permanent_uuid: "ec8c407e43b34792b91bddd88df2d715" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 37251 } }
I20260806 09:33:34.021064 14222 catalog_manager.cc:5495] T 6782134434a140adae3bcd5f43e6b2d5 P f645147ff369424ea3ef7872db16381f reported cstate change: term changed from 0 to 1, leader changed from <none> to f645147ff369424ea3ef7872db16381f (127.20.113.2). New cstate: current_term: 1 leader_uuid: "f645147ff369424ea3ef7872db16381f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e996f95d6d8a440a8d5d122cd9c2cf95" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 44583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f645147ff369424ea3ef7872db16381f" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 41673 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec8c407e43b34792b91bddd88df2d715" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 37251 } health_report { overall_health: UNKNOWN } } }
W20260806 09:33:34.066495 14855 tablet.cc:2369] T 6782134434a140adae3bcd5f43e6b2d5 P e996f95d6d8a440a8d5d122cd9c2cf95: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 09:33:34.073160 14878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:34.067800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34972 (local address 127.20.113.2:41673)
0806 09:33:34.068033 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0806 09:33:34.068039 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:33:34.068061 (+ 22us) server_negotiation.cc:407] Connection header received
0806 09:33:34.068185 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:33:34.068189 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:33:34.068290 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:33:34.068486 (+ 196us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 09:33:34.068886 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:34.069617 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:34.071002 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:34.071439 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:34.071578 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:34.072074 (+ 496us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 09:33:34.072586 (+ 512us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 09:33:34.072748 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0806 09:33:34.072810 (+ 62us) server_negotiation.cc:299] Negotiation successful
0806 09:33:34.072890 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
W20260806 09:33:34.081064 14673 tablet.cc:2369] T 6782134434a140adae3bcd5f43e6b2d5 P f645147ff369424ea3ef7872db16381f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 09:33:34.081687 14418 raft_consensus.cc:1261] T 6782134434a140adae3bcd5f43e6b2d5 P ec8c407e43b34792b91bddd88df2d715 [term 1 FOLLOWER]: Refusing update from remote peer f645147ff369424ea3ef7872db16381f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 09:33:34.081887 14796 raft_consensus.cc:1261] T 6782134434a140adae3bcd5f43e6b2d5 P e996f95d6d8a440a8d5d122cd9c2cf95 [term 1 FOLLOWER]: Refusing update from remote peer f645147ff369424ea3ef7872db16381f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 09:33:34.082865 14898 consensus_queue.cc:1035] T 6782134434a140adae3bcd5f43e6b2d5 P f645147ff369424ea3ef7872db16381f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec8c407e43b34792b91bddd88df2d715" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 37251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 09:33:34.083287 14891 consensus_queue.cc:1035] T 6782134434a140adae3bcd5f43e6b2d5 P f645147ff369424ea3ef7872db16381f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e996f95d6d8a440a8d5d122cd9c2cf95" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 44583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 09:33:34.097038 14903 mvcc.cc:205] Tried to move back new op lower bound from 7315492102468038656 to 7315492102212259840. Current Snapshot: MvccSnapshot[applied={T|T < 7315492102468038656}]
I20260806 09:33:34.099040 14905 mvcc.cc:205] Tried to move back new op lower bound from 7315492102468038656 to 7315492102212259840. Current Snapshot: MvccSnapshot[applied={T|T < 7315492102468038656}]
I20260806 09:33:34.110903 14222 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:57542:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 09:33:34.111188 14222 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:57542:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 09:33:34.114831 14222 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fef67d497b9646ce8fdcfb3d2727bed4: Sending DeleteTablet for 3 replicas of tablet 6782134434a140adae3bcd5f43e6b2d5
I20260806 09:33:34.115970 14764 tablet_service.cc:1514] Processing DeleteTablet for tablet 6782134434a140adae3bcd5f43e6b2d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:33:34 UTC) from {username='slave'} at 127.0.0.1:49202
I20260806 09:33:34.116040 14579 tablet_service.cc:1514] Processing DeleteTablet for tablet 6782134434a140adae3bcd5f43e6b2d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:33:34 UTC) from {username='slave'} at 127.0.0.1:34964
I20260806 09:33:34.116497 14389 tablet_service.cc:1514] Processing DeleteTablet for tablet 6782134434a140adae3bcd5f43e6b2d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:33:34 UTC) from {username='slave'} at 127.0.0.1:55126
I20260806 09:33:34.117036 14920 tablet_replica.cc:330] T 6782134434a140adae3bcd5f43e6b2d5 P e996f95d6d8a440a8d5d122cd9c2cf95: stopping tablet replica
I20260806 09:33:34.117547 14920 raft_consensus.cc:2229] T 6782134434a140adae3bcd5f43e6b2d5 P e996f95d6d8a440a8d5d122cd9c2cf95 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 09:33:34.117882 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 14300
I20260806 09:33:34.118192 14920 raft_consensus.cc:2258] T 6782134434a140adae3bcd5f43e6b2d5 P e996f95d6d8a440a8d5d122cd9c2cf95 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 09:33:34.126344 14920 ts_tablet_manager.cc:1905] T 6782134434a140adae3bcd5f43e6b2d5 P e996f95d6d8a440a8d5d122cd9c2cf95: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 09:33:34.127384 14921 tablet_replica.cc:330] T 6782134434a140adae3bcd5f43e6b2d5 P f645147ff369424ea3ef7872db16381f: stopping tablet replica
I20260806 09:33:34.127871 14921 raft_consensus.cc:2229] T 6782134434a140adae3bcd5f43e6b2d5 P f645147ff369424ea3ef7872db16381f [term 1 LEADER]: Raft consensus shutting down.
I20260806 09:33:34.128544 14921 raft_consensus.cc:2258] T 6782134434a140adae3bcd5f43e6b2d5 P f645147ff369424ea3ef7872db16381f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 09:33:34.130949 14921 ts_tablet_manager.cc:1905] T 6782134434a140adae3bcd5f43e6b2d5 P f645147ff369424ea3ef7872db16381f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 09:33:34.134907 14920 ts_tablet_manager.cc:1918] T 6782134434a140adae3bcd5f43e6b2d5 P e996f95d6d8a440a8d5d122cd9c2cf95: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 09:33:34.135123 14920 log.cc:1196] T 6782134434a140adae3bcd5f43e6b2d5 P e996f95d6d8a440a8d5d122cd9c2cf95: Deleting WAL directory at /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786008495763054-20932-0/minicluster-data/ts-2/wal/wals/6782134434a140adae3bcd5f43e6b2d5
I20260806 09:33:34.135768 14920 ts_tablet_manager.cc:1939] T 6782134434a140adae3bcd5f43e6b2d5 P e996f95d6d8a440a8d5d122cd9c2cf95: Deleting consensus metadata
I20260806 09:33:34.136909 14922 tablet_replica.cc:330] T 6782134434a140adae3bcd5f43e6b2d5 P ec8c407e43b34792b91bddd88df2d715: stopping tablet replica
I20260806 09:33:34.137418 14922 raft_consensus.cc:2229] T 6782134434a140adae3bcd5f43e6b2d5 P ec8c407e43b34792b91bddd88df2d715 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 09:33:34.138021 14922 raft_consensus.cc:2258] T 6782134434a140adae3bcd5f43e6b2d5 P ec8c407e43b34792b91bddd88df2d715 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 09:33:34.139014 14921 ts_tablet_manager.cc:1918] T 6782134434a140adae3bcd5f43e6b2d5 P f645147ff369424ea3ef7872db16381f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 09:33:34.139317 14921 log.cc:1196] T 6782134434a140adae3bcd5f43e6b2d5 P f645147ff369424ea3ef7872db16381f: Deleting WAL directory at /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786008495763054-20932-0/minicluster-data/ts-1/wal/wals/6782134434a140adae3bcd5f43e6b2d5
I20260806 09:33:34.139817 14205 catalog_manager.cc:4841] TS e996f95d6d8a440a8d5d122cd9c2cf95 (127.20.113.3:44583): tablet 6782134434a140adae3bcd5f43e6b2d5 (table test-table [id=3a156e706cd54692a896ccb0bfc66b12]) successfully deleted
I20260806 09:33:34.140071 14921 ts_tablet_manager.cc:1939] T 6782134434a140adae3bcd5f43e6b2d5 P f645147ff369424ea3ef7872db16381f: Deleting consensus metadata
I20260806 09:33:34.140625 14879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:34.121400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55142 (local address 127.20.113.1:37251)
0806 09:33:34.121638 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0806 09:33:34.121647 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 09:33:34.124579 (+ 2932us) server_negotiation.cc:407] Connection header received
0806 09:33:34.125592 (+ 1013us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:33:34.125599 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:33:34.125752 (+ 153us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:33:34.125842 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:33:34.127221 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:34.128434 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:34.130559 (+ 2125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:34.131227 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:34.131346 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:34.138656 (+ 7310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:33:34.138669 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:33:34.138688 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:33:34.139501 (+ 813us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:33:34.139568 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:33:34.139598 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:33:34.139703 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 09:33:34.140008 (+ 305us) server_negotiation.cc:299] Negotiation successful
0806 09:33:34.140153 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260806 09:33:34.141978 14478 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:33:34.142637 14202 catalog_manager.cc:4841] TS f645147ff369424ea3ef7872db16381f (127.20.113.2:41673): tablet 6782134434a140adae3bcd5f43e6b2d5 (table test-table [id=3a156e706cd54692a896ccb0bfc66b12]) successfully deleted
I20260806 09:33:34.142469 14922 ts_tablet_manager.cc:1905] T 6782134434a140adae3bcd5f43e6b2d5 P ec8c407e43b34792b91bddd88df2d715: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 09:33:34.230846 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 14300
W20260806 09:33:34.267653 14206 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.20.113.1:37251 (error 108)
I20260806 09:33:34.274204 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 14495
I20260806 09:33:34.275604 14880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:34.275062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.1:37251 (local address 127.0.0.1:55148)
0806 09:33:34.275253 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:34.275402 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.1:37251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260806 09:33:34.276033 14206 catalog_manager.cc:4584] TS ec8c407e43b34792b91bddd88df2d715 (127.20.113.1:37251): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6782134434a140adae3bcd5f43e6b2d5: Network error: Client connection negotiation failed: client connection to 127.20.113.1:37251: connect: Connection refused (error 111)
I20260806 09:33:34.286041 14878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:34.276623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34988 (local address 127.20.113.2:41673)
0806 09:33:34.276821 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0806 09:33:34.276826 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:33:34.276846 (+ 20us) server_negotiation.cc:407] Connection header received
0806 09:33:34.276911 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:33:34.276916 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:33:34.277013 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:33:34.277106 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:33:34.277869 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:34.279420 (+ 1551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:34.281997 (+ 2577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:34.282722 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:34.282882 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:34.284456 (+ 1574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:33:34.284464 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:33:34.284479 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:33:34.285142 (+ 663us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:33:34.285198 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:33:34.285220 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:33:34.285312 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 09:33:34.285483 (+ 171us) server_negotiation.cc:299] Negotiation successful
0806 09:33:34.285574 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260806 09:33:34.286829 14664 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:33:34.330027 14880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:34.329422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.1:37251 (local address 127.0.0.1:55164)
0806 09:33:34.329662 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:34.329776 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.1:37251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260806 09:33:34.331154 14880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:34.330722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.1:37251 (local address 127.0.0.1:55168)
0806 09:33:34.330907 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:34.331015 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.1:37251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260806 09:33:34.391676 14880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:34.391047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.1:37251 (local address 127.0.0.1:55170)
0806 09:33:34.391314 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:34.391468 (+ 154us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.1:37251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
I20260806 09:33:34.392870 14880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:34.392446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.1:37251 (local address 127.0.0.1:55180)
0806 09:33:34.392613 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:34.392711 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.1:37251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260806 09:33:34.398363 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 14495
I20260806 09:33:34.428421 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 14676
I20260806 09:33:34.435829 14874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:34.429254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49216 (local address 127.20.113.3:44583)
0806 09:33:34.429440 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0806 09:33:34.429448 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 09:33:34.429561 (+ 113us) server_negotiation.cc:407] Connection header received
0806 09:33:34.429790 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:33:34.429795 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:33:34.429887 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:33:34.429995 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:33:34.430428 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:34.431181 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:34.432892 (+ 1711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:34.433273 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:34.433405 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:34.434518 (+ 1113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:33:34.434527 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:33:34.434540 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:33:34.435087 (+ 547us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:33:34.435140 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:33:34.435159 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:33:34.435279 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0806 09:33:34.435429 (+ 150us) server_negotiation.cc:299] Negotiation successful
0806 09:33:34.435532 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 09:33:34.436540 14847 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:33:34.473543 14880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:34.472989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.1:37251 (local address 127.0.0.1:55186)
0806 09:33:34.473227 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:34.473336 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.1:37251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260806 09:33:34.474767 14880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:34.474300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.1:37251 (local address 127.0.0.1:55192)
0806 09:33:34.474539 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:34.474627 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.1:37251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129}
I20260806 09:33:34.526152 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 14676
I20260806 09:33:34.550165 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 14178
I20260806 09:33:34.551290 14262 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:33:34.657974 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 14178
2026-08-06T09:33:34Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (2975 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786008495763054-20932-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 09:33:34 dist-test-slave-45dx krb5kdc[14935](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 09:33:34 dist-test-slave-45dx krb5kdc[14935](info): set up 2 sockets
Aug 06 09:33:34 dist-test-slave-45dx krb5kdc[14935](info): commencing operation
krb5kdc: starting...
W20260806 09:33:35.965739 20932 mini_kdc.cc:121] Time spent starting KDC: real 1.256s user 0.001s sys 0.051s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786008495763054-20932-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786008495763054-20932-0/krb5kdc/test-user.keytab.
Aug 06 09:33:36 dist-test-slave-45dx krb5kdc[14935](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008816, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T09:33:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T09:33:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.113.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.113.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.113.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab.
Entry for principal kudu/127.20.113.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab.
I20260806 09:33:36.296785 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786008495763054-20932-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786008495763054-20932-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786008495763054-20932-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.113.62:45571
--webserver_interface=127.20.113.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab
--principal=kudu/127.20.113.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:46351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.113.62:45571
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:33:36.639712 14954 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:33:36.639986 14954 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:33:36.640075 14954 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:33:36.647135 14954 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 09:33:36.647234 14954 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:33:36.647289 14954 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:33:36.647346 14954 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 09:33:36.647389 14954 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 09:33:36.656522 14954 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:46351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786008495763054-20932-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786008495763054-20932-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.113.62:45571
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab
--principal=kudu/127.20.113.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.113.62:45571
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.113.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786008495763054-20932-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:33:36.657703 14954 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:33:36.658993 14954 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:33:36.670756 14965 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:33:36.670737 14962 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:33:36.670804 14961 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:33:36.676924 14954 server_base.cc:1034] running on GCE node
I20260806 09:33:36.677732 14954 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:33:36.679083 14954 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:33:36.680361 14954 hybrid_clock.cc:648] HybridClock initialized: now 1786008816680279 us; error 51 us; skew 500 ppm
I20260806 09:33:36.680711 14954 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:33:36 dist-test-slave-45dx krb5kdc[14935](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008816, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:33:36.683782 14954 init.cc:377] Logged in from keytab as kudu/127.20.113.62@KRBTEST.COM (short username kudu)
I20260806 09:33:36.685262 14954 webserver.cc:460] Webserver started at http://127.20.113.62:40757/ using document root <none> and password file <none>
I20260806 09:33:36.686007 14954 fs_manager.cc:359] Metadata directory not provided
I20260806 09:33:36.686158 14954 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:33:36.686574 14954 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:33:36.689781 14954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786008495763054-20932-0/minicluster-data/master-0/data/instance:
uuid: "479b89c2a4a24353bd05e9981e0e6548"
format_stamp: "Formatted at 2026-08-06 09:33:36 on dist-test-slave-45dx"
server_key: "db77ded8073261e6f31f7da85590823c"
server_key_iv: "19a7d0151f2d1bbae1af883e0859f3e5"
server_key_version: "encryptionkey@0"
I20260806 09:33:36.690526 14954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786008495763054-20932-0/minicluster-data/master-0/wal/instance:
uuid: "479b89c2a4a24353bd05e9981e0e6548"
format_stamp: "Formatted at 2026-08-06 09:33:36 on dist-test-slave-45dx"
server_key: "db77ded8073261e6f31f7da85590823c"
server_key_iv: "19a7d0151f2d1bbae1af883e0859f3e5"
server_key_version: "encryptionkey@0"
I20260806 09:33:36.695432 14954 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 09:33:36.698794 14978 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:33:36.700122 14954 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 09:33:36.700340 14954 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786008495763054-20932-0/minicluster-data/master-0/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786008495763054-20932-0/minicluster-data/master-0/wal
uuid: "479b89c2a4a24353bd05e9981e0e6548"
format_stamp: "Formatted at 2026-08-06 09:33:36 on dist-test-slave-45dx"
server_key: "db77ded8073261e6f31f7da85590823c"
server_key_iv: "19a7d0151f2d1bbae1af883e0859f3e5"
server_key_version: "encryptionkey@0"
I20260806 09:33:36.700555 14954 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786008495763054-20932-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786008495763054-20932-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786008495763054-20932-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:33:36.717079 14954 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:33:36.720881 14954 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:33:36.721156 14954 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:33:36.739741 14954 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.62:45571
I20260806 09:33:36.739764 15047 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.62:45571 every 8 connection(s)
I20260806 09:33:36.741114 14954 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
I20260806 09:33:36.744829 15049 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:33:36.747826 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 14954
I20260806 09:33:36.748039 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786008495763054-20932-0/minicluster-data/master-0/wal/instance
I20260806 09:33:36.748590 20932 external_mini_cluster.cc:1428] Setting key f15df4f22d184bccd93557827fbaa816
I20260806 09:33:36.757083 15049 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 479b89c2a4a24353bd05e9981e0e6548: Bootstrap starting.
I20260806 09:33:36.760797 15049 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 479b89c2a4a24353bd05e9981e0e6548: Neither blocks nor log segments found. Creating new log.
I20260806 09:33:36.762105 15049 log.cc:824] T 00000000000000000000000000000000 P 479b89c2a4a24353bd05e9981e0e6548: Log is configured to *not* fsync() on all Append() calls
Aug 06 09:33:36 dist-test-slave-45dx krb5kdc[14935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008816, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.113.62@KRBTEST.COM
I20260806 09:33:36.764895 15049 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 479b89c2a4a24353bd05e9981e0e6548: No bootstrap required, opened a new log
I20260806 09:33:36.772817 15053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:36.750418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34964 (local address 127.20.113.62:45571)
0806 09:33:36.751245 (+ 827us) server_negotiation.cc:206] Beginning negotiation
0806 09:33:36.751255 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 09:33:36.751293 (+ 38us) server_negotiation.cc:407] Connection header received
0806 09:33:36.752999 (+ 1706us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:33:36.753020 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:33:36.753377 (+ 357us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:33:36.753677 (+ 300us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:33:36.755513 (+ 1836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:36.756821 (+ 1308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:36.758773 (+ 1952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:36.759412 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:36.759532 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:36.764478 (+ 4946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:33:36.764488 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:33:36.764506 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:33:36.764545 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:33:36.768511 (+ 3966us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:33:36.769218 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:33:36.769225 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:33:36.769233 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:33:36.769460 (+ 227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:33:36.769845 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:33:36.769850 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:33:36.769853 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:33:36.770556 (+ 703us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:33:36.770871 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0806 09:33:36.771144 (+ 273us) server_negotiation.cc:299] Negotiation successful
0806 09:33:36.771562 (+ 418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":637,"thread_start_us":415,"threads_started":1}
I20260806 09:33:36.775219 15049 raft_consensus.cc:348] T 00000000000000000000000000000000 P 479b89c2a4a24353bd05e9981e0e6548 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "479b89c2a4a24353bd05e9981e0e6548" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 45571 } }
I20260806 09:33:36.775714 15049 raft_consensus.cc:374] T 00000000000000000000000000000000 P 479b89c2a4a24353bd05e9981e0e6548 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:33:36.775813 15049 raft_consensus.cc:729] T 00000000000000000000000000000000 P 479b89c2a4a24353bd05e9981e0e6548 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 479b89c2a4a24353bd05e9981e0e6548, State: Initialized, Role: FOLLOWER
I20260806 09:33:36.776414 15049 consensus_queue.cc:260] T 00000000000000000000000000000000 P 479b89c2a4a24353bd05e9981e0e6548 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "479b89c2a4a24353bd05e9981e0e6548" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 45571 } }
I20260806 09:33:36.776649 15049 raft_consensus.cc:388] T 00000000000000000000000000000000 P 479b89c2a4a24353bd05e9981e0e6548 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 09:33:36.776836 15049 raft_consensus.cc:482] T 00000000000000000000000000000000 P 479b89c2a4a24353bd05e9981e0e6548 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 09:33:36.777016 15049 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 479b89c2a4a24353bd05e9981e0e6548 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:33:36.779109 15049 raft_consensus.cc:504] T 00000000000000000000000000000000 P 479b89c2a4a24353bd05e9981e0e6548 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "479b89c2a4a24353bd05e9981e0e6548" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 45571 } }
I20260806 09:33:36.779567 15049 leader_election.cc:302] T 00000000000000000000000000000000 P 479b89c2a4a24353bd05e9981e0e6548 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 479b89c2a4a24353bd05e9981e0e6548; no voters:
I20260806 09:33:36.780258 15049 leader_election.cc:288] T 00000000000000000000000000000000 P 479b89c2a4a24353bd05e9981e0e6548 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 09:33:36.780438 15056 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 479b89c2a4a24353bd05e9981e0e6548 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:33:36.781611 15056 raft_consensus.cc:686] T 00000000000000000000000000000000 P 479b89c2a4a24353bd05e9981e0e6548 [term 1 LEADER]: Becoming Leader. State: Replica: 479b89c2a4a24353bd05e9981e0e6548, State: Running, Role: LEADER
I20260806 09:33:36.782094 15056 consensus_queue.cc:237] T 00000000000000000000000000000000 P 479b89c2a4a24353bd05e9981e0e6548 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "479b89c2a4a24353bd05e9981e0e6548" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 45571 } }
I20260806 09:33:36.782421 15049 sys_catalog.cc:564] T 00000000000000000000000000000000 P 479b89c2a4a24353bd05e9981e0e6548 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 09:33:36.785081 15059 sys_catalog.cc:455] T 00000000000000000000000000000000 P 479b89c2a4a24353bd05e9981e0e6548 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 479b89c2a4a24353bd05e9981e0e6548. Latest consensus state: current_term: 1 leader_uuid: "479b89c2a4a24353bd05e9981e0e6548" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "479b89c2a4a24353bd05e9981e0e6548" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 45571 } } }
I20260806 09:33:36.785183 15057 sys_catalog.cc:455] T 00000000000000000000000000000000 P 479b89c2a4a24353bd05e9981e0e6548 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "479b89c2a4a24353bd05e9981e0e6548" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "479b89c2a4a24353bd05e9981e0e6548" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 45571 } } }
I20260806 09:33:36.785524 15059 sys_catalog.cc:458] T 00000000000000000000000000000000 P 479b89c2a4a24353bd05e9981e0e6548 [sys.catalog]: This master's current role is: LEADER
I20260806 09:33:36.785547 15057 sys_catalog.cc:458] T 00000000000000000000000000000000 P 479b89c2a4a24353bd05e9981e0e6548 [sys.catalog]: This master's current role is: LEADER
I20260806 09:33:36.788935 15065 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 09:33:36.793525 15065 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 09:33:36.801904 15065 catalog_manager.cc:1347] Generated new cluster ID: 560a6fcdacf84132bcd892c919408db2
I20260806 09:33:36.802027 15065 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 09:33:36.809949 15065 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 09:33:36.811124 15065 catalog_manager.cc:1504] Loading token signing keys...
I20260806 09:33:36.825961 15065 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 479b89c2a4a24353bd05e9981e0e6548: Generated new TSK 0
I20260806 09:33:36.826840 15065 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.113.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.113.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.113.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.1.keytab.
Entry for principal kudu/127.20.113.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.1.keytab.
I20260806 09:33:36.913550 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786008495763054-20932-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786008495763054-20932-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786008495763054-20932-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786008495763054-20932-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.113.1:0
--local_ip_for_outbound_sockets=127.20.113.1
--webserver_interface=127.20.113.1
--webserver_port=0
--tserver_master_addrs=127.20.113.62:45571
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.1.keytab
--principal=kudu/127.20.113.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:46351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:33:37.287907 15083 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:33:37.288364 15083 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:33:37.288483 15083 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:33:37.295359 15083 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:33:37.295500 15083 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:33:37.295662 15083 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.113.1
I20260806 09:33:37.305518 15083 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:46351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786008495763054-20932-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786008495763054-20932-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.1.keytab
--principal=kudu/127.20.113.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.113.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786008495763054-20932-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.113.1
--webserver_port=0
--tserver_master_addrs=127.20.113.62:45571
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.113.1
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786008495763054-20932-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:33:37.306842 15083 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:33:37.308189 15083 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:33:37.322091 15088 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:33:37.322014 15092 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:33:37.322520 15090 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:33:37.323433 15083 server_base.cc:1034] running on GCE node
I20260806 09:33:37.324141 15083 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:33:37.325181 15083 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:33:37.326428 15083 hybrid_clock.cc:648] HybridClock initialized: now 1786008817326337 us; error 79 us; skew 500 ppm
I20260806 09:33:37.326750 15083 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:33:37 dist-test-slave-45dx krb5kdc[14935](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008817, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:33:37.330084 15083 init.cc:377] Logged in from keytab as kudu/127.20.113.1@KRBTEST.COM (short username kudu)
I20260806 09:33:37.331550 15083 webserver.cc:460] Webserver started at http://127.20.113.1:36205/ using document root <none> and password file <none>
I20260806 09:33:37.332345 15083 fs_manager.cc:359] Metadata directory not provided
I20260806 09:33:37.332510 15083 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:33:37.332865 15083 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:33:37.335866 15083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786008495763054-20932-0/minicluster-data/ts-0/data/instance:
uuid: "61db43ea12324acfa22105bb59564a75"
format_stamp: "Formatted at 2026-08-06 09:33:37 on dist-test-slave-45dx"
server_key: "1485b5100074275e170d3dd12d5e0da0"
server_key_iv: "23683819616f9640fbc12b911a490b59"
server_key_version: "encryptionkey@0"
I20260806 09:33:37.336717 15083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786008495763054-20932-0/minicluster-data/ts-0/wal/instance:
uuid: "61db43ea12324acfa22105bb59564a75"
format_stamp: "Formatted at 2026-08-06 09:33:37 on dist-test-slave-45dx"
server_key: "1485b5100074275e170d3dd12d5e0da0"
server_key_iv: "23683819616f9640fbc12b911a490b59"
server_key_version: "encryptionkey@0"
I20260806 09:33:37.342049 15083 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 09:33:37.345734 15111 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:33:37.347164 15083 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 09:33:37.347349 15083 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786008495763054-20932-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786008495763054-20932-0/minicluster-data/ts-0/wal
uuid: "61db43ea12324acfa22105bb59564a75"
format_stamp: "Formatted at 2026-08-06 09:33:37 on dist-test-slave-45dx"
server_key: "1485b5100074275e170d3dd12d5e0da0"
server_key_iv: "23683819616f9640fbc12b911a490b59"
server_key_version: "encryptionkey@0"
I20260806 09:33:37.347559 15083 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786008495763054-20932-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786008495763054-20932-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786008495763054-20932-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:33:37.371330 15083 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:33:37.375411 15083 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:33:37.375725 15083 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:33:37.376708 15083 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:33:37.378064 15083 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:33:37.378171 15083 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:33:37.378330 15083 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:33:37.378418 15083 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:33:37.413002 15083 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.1:41215
I20260806 09:33:37.413089 15272 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.1:41215 every 8 connection(s)
I20260806 09:33:37.414330 15083 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786008495763054-20932-0/minicluster-data/ts-0/data/info.pb
I20260806 09:33:37.423940 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 15083
I20260806 09:33:37.424176 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786008495763054-20932-0/minicluster-data/ts-0/wal/instance
I20260806 09:33:37.424850 20932 external_mini_cluster.cc:1428] Setting key 3eaf9f3a2a5e0d743d2717fb0774278a
Aug 06 09:33:37 dist-test-slave-45dx krb5kdc[14935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008817, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.1@KRBTEST.COM for kudu/127.20.113.62@KRBTEST.COM
I20260806 09:33:37.436085 15277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:37.417054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.1:58491 (local address 127.20.113.62:45571)
0806 09:33:37.417534 (+ 480us) server_negotiation.cc:206] Beginning negotiation
0806 09:33:37.417542 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 09:33:37.418510 (+ 968us) server_negotiation.cc:407] Connection header received
0806 09:33:37.420268 (+ 1758us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:33:37.420272 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:33:37.420401 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:33:37.420515 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:33:37.421989 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:37.422724 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:37.424560 (+ 1836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:37.425175 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:37.425367 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:37.429196 (+ 3829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:33:37.429205 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:33:37.429213 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:33:37.429259 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:33:37.432897 (+ 3638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:33:37.433608 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:33:37.433616 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:33:37.433621 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:33:37.433775 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:33:37.434181 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:33:37.434188 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:33:37.434193 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:33:37.434576 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:33:37.434736 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0806 09:33:37.435381 (+ 645us) server_negotiation.cc:299] Negotiation successful
0806 09:33:37.435634 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":225,"threads_started":1}
I20260806 09:33:37.436688 15278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:37.417620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:45571 (local address 127.20.113.1:58491)
0806 09:33:37.418347 (+ 727us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:37.418394 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 09:33:37.420019 (+ 1625us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:33:37.420716 (+ 697us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:33:37.420723 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:33:37.421237 (+ 514us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:33:37.421825 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:33:37.421843 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:33:37.422894 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:33:37.422899 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:33:37.424395 (+ 1496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:33:37.424408 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:33:37.425481 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:33:37.425491 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:33:37.425788 (+ 297us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:37.426584 (+ 796us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:33:37.426615 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:33:37.428965 (+ 2350us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:33:37.433036 (+ 4071us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:33:37.433043 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:33:37.433054 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:33:37.433487 (+ 433us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:33:37.433891 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:33:37.433894 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:33:37.433897 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:33:37.434073 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:33:37.434706 (+ 633us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:33:37.434712 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:33:37.435001 (+ 289us) client_negotiation.cc:769] Sending connection context
0806 09:33:37.435236 (+ 235us) client_negotiation.cc:240] Negotiation successful
0806 09:33:37.435601 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":299,"threads_started":1}
I20260806 09:33:37.438273 15273 heartbeater.cc:347] Connected to a master server at 127.20.113.62:45571
I20260806 09:33:37.438589 15273 heartbeater.cc:464] Registering TS with master...
I20260806 09:33:37.439287 15273 heartbeater.cc:511] Master 127.20.113.62:45571 requested a full tablet report, sending...
I20260806 09:33:37.441565 14999 ts_manager.cc:194] Registered new tserver with Master: 61db43ea12324acfa22105bb59564a75 (127.20.113.1:41215)
I20260806 09:33:37.443658 14999 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.113.1@KRBTEST.COM'} at 127.20.113.1:58491
WARNING: no policy specified for kudu/127.20.113.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.113.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.113.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.2.keytab.
Entry for principal kudu/127.20.113.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.2.keytab.
I20260806 09:33:37.501765 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786008495763054-20932-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786008495763054-20932-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786008495763054-20932-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786008495763054-20932-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.113.2:0
--local_ip_for_outbound_sockets=127.20.113.2
--webserver_interface=127.20.113.2
--webserver_port=0
--tserver_master_addrs=127.20.113.62:45571
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.2.keytab
--principal=kudu/127.20.113.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:46351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:33:37.845715 15282 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:33:37.845979 15282 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:33:37.846072 15282 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:33:37.853093 15282 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:33:37.853197 15282 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:33:37.853300 15282 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.113.2
I20260806 09:33:37.862373 15282 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:46351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786008495763054-20932-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786008495763054-20932-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.2.keytab
--principal=kudu/127.20.113.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.113.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786008495763054-20932-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.113.2
--webserver_port=0
--tserver_master_addrs=127.20.113.62:45571
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.113.2
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786008495763054-20932-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:33:37.863510 15282 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:33:37.864763 15282 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:33:37.877653 15292 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:33:37.877745 15295 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:33:37.877628 15300 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:33:37.878929 15282 server_base.cc:1034] running on GCE node
I20260806 09:33:37.879523 15282 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:33:37.880455 15282 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:33:37.881680 15282 hybrid_clock.cc:648] HybridClock initialized: now 1786008817881638 us; error 50 us; skew 500 ppm
I20260806 09:33:37.882004 15282 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:33:37 dist-test-slave-45dx krb5kdc[14935](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008817, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:33:37.885167 15282 init.cc:377] Logged in from keytab as kudu/127.20.113.2@KRBTEST.COM (short username kudu)
I20260806 09:33:37.886495 15282 webserver.cc:460] Webserver started at http://127.20.113.2:37057/ using document root <none> and password file <none>
I20260806 09:33:37.887181 15282 fs_manager.cc:359] Metadata directory not provided
I20260806 09:33:37.887333 15282 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:33:37.887665 15282 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:33:37.890554 15282 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786008495763054-20932-0/minicluster-data/ts-1/data/instance:
uuid: "5f395e3ef35d4d0f99886d8de3168f8d"
format_stamp: "Formatted at 2026-08-06 09:33:37 on dist-test-slave-45dx"
server_key: "4fb0bb6e89419f34693cec2c1459b9a1"
server_key_iv: "7074ef86cb4d245b94a808aecd7ac339"
server_key_version: "encryptionkey@0"
I20260806 09:33:37.891291 15282 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786008495763054-20932-0/minicluster-data/ts-1/wal/instance:
uuid: "5f395e3ef35d4d0f99886d8de3168f8d"
format_stamp: "Formatted at 2026-08-06 09:33:37 on dist-test-slave-45dx"
server_key: "4fb0bb6e89419f34693cec2c1459b9a1"
server_key_iv: "7074ef86cb4d245b94a808aecd7ac339"
server_key_version: "encryptionkey@0"
I20260806 09:33:37.896250 15282 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260806 09:33:37.899601 15311 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:33:37.900913 15282 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 09:33:37.901087 15282 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786008495763054-20932-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786008495763054-20932-0/minicluster-data/ts-1/wal
uuid: "5f395e3ef35d4d0f99886d8de3168f8d"
format_stamp: "Formatted at 2026-08-06 09:33:37 on dist-test-slave-45dx"
server_key: "4fb0bb6e89419f34693cec2c1459b9a1"
server_key_iv: "7074ef86cb4d245b94a808aecd7ac339"
server_key_version: "encryptionkey@0"
I20260806 09:33:37.901280 15282 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786008495763054-20932-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786008495763054-20932-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786008495763054-20932-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:33:37.941007 15282 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:33:37.944936 15282 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:33:37.945219 15282 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:33:37.946244 15282 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:33:37.947573 15282 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:33:37.947677 15282 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:33:37.947824 15282 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:33:37.947896 15282 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:33:37.981117 15282 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.2:42985
I20260806 09:33:37.981173 15498 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.2:42985 every 8 connection(s)
I20260806 09:33:37.982434 15282 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786008495763054-20932-0/minicluster-data/ts-1/data/info.pb
I20260806 09:33:37.987025 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 15282
I20260806 09:33:37.987242 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786008495763054-20932-0/minicluster-data/ts-1/wal/instance
I20260806 09:33:37.987838 20932 external_mini_cluster.cc:1428] Setting key 659a9144a36bb51e4316c6063e73938b
Aug 06 09:33:37 dist-test-slave-45dx krb5kdc[14935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008817, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.2@KRBTEST.COM for kudu/127.20.113.62@KRBTEST.COM
I20260806 09:33:38.006860 15501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:37.985509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.2:53411 (local address 127.20.113.62:45571)
0806 09:33:37.985979 (+ 470us) server_negotiation.cc:206] Beginning negotiation
0806 09:33:37.985984 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:33:37.987008 (+ 1024us) server_negotiation.cc:407] Connection header received
0806 09:33:37.989451 (+ 2443us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:33:37.989456 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:33:37.989535 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:33:37.989666 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:33:37.991661 (+ 1995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:37.992426 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:37.994826 (+ 2400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:37.995236 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:37.995346 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:37.999552 (+ 4206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:33:37.999561 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:33:37.999569 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:33:37.999614 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:33:38.003244 (+ 3630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:33:38.004010 (+ 766us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:33:38.004018 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:33:38.004023 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:33:38.004189 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:33:38.004656 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:33:38.004680 (+ 24us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:33:38.004686 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:33:38.005109 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:33:38.005283 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0806 09:33:38.006227 (+ 944us) server_negotiation.cc:299] Negotiation successful
0806 09:33:38.006440 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":267,"threads_started":1}
I20260806 09:33:38.007897 15503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:37.985983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:45571 (local address 127.20.113.2:53411)
0806 09:33:37.986744 (+ 761us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:37.986831 (+ 87us) client_negotiation.cc:174] Beginning negotiation
0806 09:33:37.989143 (+ 2312us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:33:37.989929 (+ 786us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:33:37.989938 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:33:37.990720 (+ 782us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:33:37.991493 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:33:37.991520 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:33:37.992594 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:33:37.992602 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:33:37.994611 (+ 2009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:33:37.994628 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:33:37.995472 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:33:37.995481 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:33:37.995775 (+ 294us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:37.996578 (+ 803us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:33:37.996611 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:33:37.999324 (+ 2713us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:33:38.003459 (+ 4135us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:33:38.003465 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:33:38.003476 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:33:38.003884 (+ 408us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:33:38.004296 (+ 412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:33:38.004299 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:33:38.004341 (+ 42us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:33:38.004543 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:33:38.005241 (+ 698us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:33:38.005247 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:33:38.005640 (+ 393us) client_negotiation.cc:769] Sending connection context
0806 09:33:38.005916 (+ 276us) client_negotiation.cc:240] Negotiation successful
0806 09:33:38.006437 (+ 521us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":280,"threads_started":1}
I20260806 09:33:38.009548 15499 heartbeater.cc:347] Connected to a master server at 127.20.113.62:45571
I20260806 09:33:38.009891 15499 heartbeater.cc:464] Registering TS with master...
I20260806 09:33:38.010753 15499 heartbeater.cc:511] Master 127.20.113.62:45571 requested a full tablet report, sending...
I20260806 09:33:38.012391 14999 ts_manager.cc:194] Registered new tserver with Master: 5f395e3ef35d4d0f99886d8de3168f8d (127.20.113.2:42985)
I20260806 09:33:38.013648 14999 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.113.2@KRBTEST.COM'} at 127.20.113.2:53411
WARNING: no policy specified for kudu/127.20.113.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.113.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.113.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.3.keytab.
Entry for principal kudu/127.20.113.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.3.keytab.
I20260806 09:33:38.066833 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786008495763054-20932-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786008495763054-20932-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786008495763054-20932-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786008495763054-20932-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.113.3:0
--local_ip_for_outbound_sockets=127.20.113.3
--webserver_interface=127.20.113.3
--webserver_port=0
--tserver_master_addrs=127.20.113.62:45571
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.3.keytab
--principal=kudu/127.20.113.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:46351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:33:38.413209 15506 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:33:38.413548 15506 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:33:38.413744 15506 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:33:38.420950 15506 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:33:38.421092 15506 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:33:38.421274 15506 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.113.3
I20260806 09:33:38.431268 15506 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:46351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786008495763054-20932-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786008495763054-20932-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.3.keytab
--principal=kudu/127.20.113.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.113.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786008495763054-20932-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.113.3
--webserver_port=0
--tserver_master_addrs=127.20.113.62:45571
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.113.3
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786008495763054-20932-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:33:38.432641 15506 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:33:38.433974 15506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 09:33:38.446641 15273 heartbeater.cc:503] Master 127.20.113.62:45571 was elected leader, sending a full tablet report...
W20260806 09:33:38.448588 15517 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:33:38.448653 15520 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:33:38.449014 15514 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:33:38.449518 15506 server_base.cc:1034] running on GCE node
I20260806 09:33:38.450059 15506 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:33:38.450979 15506 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:33:38.452169 15506 hybrid_clock.cc:648] HybridClock initialized: now 1786008818452095 us; error 71 us; skew 500 ppm
I20260806 09:33:38.452478 15506 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:33:38 dist-test-slave-45dx krb5kdc[14935](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008818, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:33:38.455585 15506 init.cc:377] Logged in from keytab as kudu/127.20.113.3@KRBTEST.COM (short username kudu)
I20260806 09:33:38.457098 15506 webserver.cc:460] Webserver started at http://127.20.113.3:42255/ using document root <none> and password file <none>
I20260806 09:33:38.457768 15506 fs_manager.cc:359] Metadata directory not provided
I20260806 09:33:38.457907 15506 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:33:38.458252 15506 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:33:38.461160 15506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786008495763054-20932-0/minicluster-data/ts-2/data/instance:
uuid: "c5ae52971ee4440ebf20daf5b3223d59"
format_stamp: "Formatted at 2026-08-06 09:33:38 on dist-test-slave-45dx"
server_key: "5cc0620b833e259732bc40cb82128031"
server_key_iv: "a133a5ada2c0d61cb59bfa9f369932c3"
server_key_version: "encryptionkey@0"
I20260806 09:33:38.461947 15506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786008495763054-20932-0/minicluster-data/ts-2/wal/instance:
uuid: "c5ae52971ee4440ebf20daf5b3223d59"
format_stamp: "Formatted at 2026-08-06 09:33:38 on dist-test-slave-45dx"
server_key: "5cc0620b833e259732bc40cb82128031"
server_key_iv: "a133a5ada2c0d61cb59bfa9f369932c3"
server_key_version: "encryptionkey@0"
I20260806 09:33:38.467028 15506 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 09:33:38.470431 15531 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:33:38.471678 15506 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 09:33:38.471854 15506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786008495763054-20932-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786008495763054-20932-0/minicluster-data/ts-2/wal
uuid: "c5ae52971ee4440ebf20daf5b3223d59"
format_stamp: "Formatted at 2026-08-06 09:33:38 on dist-test-slave-45dx"
server_key: "5cc0620b833e259732bc40cb82128031"
server_key_iv: "a133a5ada2c0d61cb59bfa9f369932c3"
server_key_version: "encryptionkey@0"
I20260806 09:33:38.472052 15506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786008495763054-20932-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786008495763054-20932-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786008495763054-20932-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:33:38.520752 15506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:33:38.524981 15506 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:33:38.525288 15506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:33:38.526332 15506 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:33:38.527827 15506 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:33:38.527935 15506 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:33:38.528093 15506 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:33:38.528178 15506 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:33:38.562755 15506 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.3:33367
I20260806 09:33:38.562813 15689 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.3:33367 every 8 connection(s)
I20260806 09:33:38.564045 15506 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786008495763054-20932-0/minicluster-data/ts-2/data/info.pb
I20260806 09:33:38.570441 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 15506
I20260806 09:33:38.570626 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786008495763054-20932-0/minicluster-data/ts-2/wal/instance
I20260806 09:33:38.571071 20932 external_mini_cluster.cc:1428] Setting key 76ea4821a9140fbd18966ae1a838aa1b
Aug 06 09:33:38 dist-test-slave-45dx krb5kdc[14935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008818, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.3@KRBTEST.COM for kudu/127.20.113.62@KRBTEST.COM
I20260806 09:33:38.583776 15692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:38.566601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.3:37083 (local address 127.20.113.62:45571)
0806 09:33:38.567136 (+ 535us) server_negotiation.cc:206] Beginning negotiation
0806 09:33:38.567141 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:33:38.567956 (+ 815us) server_negotiation.cc:407] Connection header received
0806 09:33:38.569859 (+ 1903us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:33:38.569864 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:33:38.569978 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:33:38.570097 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:33:38.571699 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:38.572691 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:38.574170 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:38.574569 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:38.574700 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:38.578052 (+ 3352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:33:38.578063 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:33:38.578078 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:33:38.578109 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:33:38.580610 (+ 2501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:33:38.581297 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:33:38.581307 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:33:38.581310 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:33:38.581430 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:33:38.581904 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:33:38.581909 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:33:38.581912 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:33:38.582175 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:33:38.582374 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0806 09:33:38.583103 (+ 729us) server_negotiation.cc:299] Negotiation successful
0806 09:33:38.583321 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":428,"thread_start_us":303,"threads_started":1}
I20260806 09:33:38.584663 15693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:38.567113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:45571 (local address 127.20.113.3:37083)
0806 09:33:38.567804 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:38.567842 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 09:33:38.569590 (+ 1748us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:33:38.570331 (+ 741us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:33:38.570339 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:33:38.570941 (+ 602us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:33:38.571530 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:33:38.571548 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:33:38.572855 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:33:38.572858 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:33:38.574025 (+ 1167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:33:38.574037 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:33:38.574802 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:33:38.574809 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:33:38.575025 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:38.575610 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:33:38.575636 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:33:38.577848 (+ 2212us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:33:38.580748 (+ 2900us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:33:38.580756 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:33:38.580770 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:33:38.581170 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:33:38.581559 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:33:38.581564 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:33:38.581567 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:33:38.581776 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:33:38.582314 (+ 538us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:33:38.582324 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:33:38.582694 (+ 370us) client_negotiation.cc:769] Sending connection context
0806 09:33:38.582966 (+ 272us) client_negotiation.cc:240] Negotiation successful
0806 09:33:38.583496 (+ 530us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":282,"threads_started":1}
I20260806 09:33:38.586004 15690 heartbeater.cc:347] Connected to a master server at 127.20.113.62:45571
I20260806 09:33:38.586330 15690 heartbeater.cc:464] Registering TS with master...
I20260806 09:33:38.587028 15690 heartbeater.cc:511] Master 127.20.113.62:45571 requested a full tablet report, sending...
I20260806 09:33:38.588457 14999 ts_manager.cc:194] Registered new tserver with Master: c5ae52971ee4440ebf20daf5b3223d59 (127.20.113.3:33367)
I20260806 09:33:38.589246 14999 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.113.3@KRBTEST.COM'} at 127.20.113.3:37083
I20260806 09:33:38.598320 20932 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 09:33:38.615684 15692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:38.602524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34972 (local address 127.20.113.62:45571)
0806 09:33:38.602682 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0806 09:33:38.602686 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 09:33:38.603237 (+ 551us) server_negotiation.cc:407] Connection header received
0806 09:33:38.603372 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:33:38.603375 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:33:38.603485 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:33:38.603569 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:33:38.605203 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:38.605956 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:38.607427 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:38.607840 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:38.607941 (+ 101us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:38.610658 (+ 2717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:33:38.610663 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:33:38.610666 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:33:38.610697 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:33:38.613317 (+ 2620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:33:38.614034 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:33:38.614038 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:33:38.614040 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:33:38.614153 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:33:38.614560 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:33:38.614563 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:33:38.614565 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:33:38.614836 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:33:38.615037 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0806 09:33:38.615334 (+ 297us) server_negotiation.cc:299] Negotiation successful
0806 09:33:38.615465 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 09:33:38.619719 14999 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34972:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 09:33:38.626672 14999 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 09:33:38.660399 15709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:38.638788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60112 (local address 127.20.113.2:42985)
0806 09:33:38.651821 (+ 13033us) server_negotiation.cc:206] Beginning negotiation
0806 09:33:38.651829 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 09:33:38.651861 (+ 32us) server_negotiation.cc:407] Connection header received
0806 09:33:38.651970 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:33:38.651979 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:33:38.652295 (+ 316us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:33:38.652547 (+ 252us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:33:38.653233 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:38.654741 (+ 1508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:38.657762 (+ 3021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:38.659104 (+ 1342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:38.659253 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:38.659482 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0806 09:33:38.659824 (+ 342us) server_negotiation.cc:299] Negotiation successful
0806 09:33:38.660002 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12885,"thread_start_us":12742,"threads_started":1}
I20260806 09:33:38.660493 15707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:38.638652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.2:42985 (local address 127.0.0.1:60112)
0806 09:33:38.639819 (+ 1167us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:38.639872 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 09:33:38.640041 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:33:38.652583 (+ 12542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:33:38.652590 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:33:38.652620 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:33:38.653032 (+ 412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:33:38.653048 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:33:38.654895 (+ 1847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:33:38.654901 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:33:38.657602 (+ 2701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:33:38.657615 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:33:38.659327 (+ 1712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:33:38.659336 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:33:38.659552 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:38.659604 (+ 52us) client_negotiation.cc:769] Sending connection context
0806 09:33:38.659759 (+ 155us) client_negotiation.cc:240] Negotiation successful
0806 09:33:38.659967 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":909,"mutex_wait_us":101,"thread_start_us":1614,"threads_started":1}
I20260806 09:33:38.661329 15714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:38.642857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38558 (local address 127.20.113.1:41215)
0806 09:33:38.651180 (+ 8323us) server_negotiation.cc:206] Beginning negotiation
0806 09:33:38.651192 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 09:33:38.652405 (+ 1213us) server_negotiation.cc:407] Connection header received
0806 09:33:38.652742 (+ 337us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:33:38.652750 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:33:38.652942 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:33:38.653127 (+ 185us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:33:38.654066 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:38.655267 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:38.658361 (+ 3094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:38.659388 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:38.659530 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:38.659777 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0806 09:33:38.660914 (+ 1137us) server_negotiation.cc:299] Negotiation successful
0806 09:33:38.661069 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8123,"thread_start_us":7962,"threads_started":1}
I20260806 09:33:38.661875 15713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:38.641197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.1:41215 (local address 127.0.0.1:38558)
0806 09:33:38.652268 (+ 11071us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:38.652287 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 09:33:38.652526 (+ 239us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:33:38.653410 (+ 884us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:33:38.653416 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:33:38.653447 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:33:38.653921 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:33:38.653930 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:33:38.655431 (+ 1501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:33:38.655436 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:33:38.658132 (+ 2696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:33:38.658153 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:33:38.659925 (+ 1772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:33:38.659931 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:33:38.660096 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:38.660135 (+ 39us) client_negotiation.cc:769] Sending connection context
0806 09:33:38.660227 (+ 92us) client_negotiation.cc:240] Negotiation successful
0806 09:33:38.660370 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10938,"mutex_wait_us":66,"thread_start_us":10665,"threads_started":1}
I20260806 09:33:38.662240 15712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:38.639611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52152 (local address 127.20.113.3:33367)
0806 09:33:38.650448 (+ 10837us) server_negotiation.cc:206] Beginning negotiation
0806 09:33:38.650458 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 09:33:38.652529 (+ 2071us) server_negotiation.cc:407] Connection header received
0806 09:33:38.652631 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:33:38.652651 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:33:38.652893 (+ 242us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:33:38.653088 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:33:38.653867 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:38.655274 (+ 1407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:38.658306 (+ 3032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:38.659699 (+ 1393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:38.659846 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:38.660074 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0806 09:33:38.661664 (+ 1590us) server_negotiation.cc:299] Negotiation successful
0806 09:33:38.661820 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10632,"thread_start_us":10465,"threads_started":1}
I20260806 09:33:38.662930 15710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:38.639205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.3:33367 (local address 127.0.0.1:52152)
0806 09:33:38.652170 (+ 12965us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:38.652193 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 09:33:38.652379 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:33:38.653110 (+ 731us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:33:38.653115 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:33:38.653139 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:33:38.653639 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:33:38.653654 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:33:38.655467 (+ 1813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:33:38.655473 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:33:38.658153 (+ 2680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:33:38.658163 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:33:38.660230 (+ 2067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:33:38.660236 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:33:38.660457 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:38.660509 (+ 52us) client_negotiation.cc:769] Sending connection context
0806 09:33:38.660613 (+ 104us) client_negotiation.cc:240] Negotiation successful
0806 09:33:38.662523 (+ 1910us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12812,"mutex_wait_us":102,"thread_start_us":11238,"threads_started":1}
I20260806 09:33:38.664635 15390 tablet_service.cc:1467] Processing CreateTablet for tablet 41fe54925c9a458183f7e5cf2649c40b (DEFAULT_TABLE table=test-table [id=fa7454d334124c65ae7728a1bfb87502]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:33:38.665349 15172 tablet_service.cc:1467] Processing CreateTablet for tablet 41fe54925c9a458183f7e5cf2649c40b (DEFAULT_TABLE table=test-table [id=fa7454d334124c65ae7728a1bfb87502]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:33:38.666301 15390 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41fe54925c9a458183f7e5cf2649c40b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:33:38.666924 15172 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41fe54925c9a458183f7e5cf2649c40b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:33:38.668467 15597 tablet_service.cc:1467] Processing CreateTablet for tablet 41fe54925c9a458183f7e5cf2649c40b (DEFAULT_TABLE table=test-table [id=fa7454d334124c65ae7728a1bfb87502]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:33:38.670660 15597 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41fe54925c9a458183f7e5cf2649c40b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:33:38.678390 15715 tablet_bootstrap.cc:492] T 41fe54925c9a458183f7e5cf2649c40b P 5f395e3ef35d4d0f99886d8de3168f8d: Bootstrap starting.
I20260806 09:33:38.681594 15715 tablet_bootstrap.cc:654] T 41fe54925c9a458183f7e5cf2649c40b P 5f395e3ef35d4d0f99886d8de3168f8d: Neither blocks nor log segments found. Creating new log.
I20260806 09:33:38.682839 15716 tablet_bootstrap.cc:492] T 41fe54925c9a458183f7e5cf2649c40b P 61db43ea12324acfa22105bb59564a75: Bootstrap starting.
I20260806 09:33:38.683056 15715 log.cc:824] T 41fe54925c9a458183f7e5cf2649c40b P 5f395e3ef35d4d0f99886d8de3168f8d: Log is configured to *not* fsync() on all Append() calls
I20260806 09:33:38.685842 15718 tablet_bootstrap.cc:492] T 41fe54925c9a458183f7e5cf2649c40b P c5ae52971ee4440ebf20daf5b3223d59: Bootstrap starting.
I20260806 09:33:38.686563 15715 tablet_bootstrap.cc:492] T 41fe54925c9a458183f7e5cf2649c40b P 5f395e3ef35d4d0f99886d8de3168f8d: No bootstrap required, opened a new log
I20260806 09:33:38.686790 15716 tablet_bootstrap.cc:654] T 41fe54925c9a458183f7e5cf2649c40b P 61db43ea12324acfa22105bb59564a75: Neither blocks nor log segments found. Creating new log.
I20260806 09:33:38.686923 15715 ts_tablet_manager.cc:1397] T 41fe54925c9a458183f7e5cf2649c40b P 5f395e3ef35d4d0f99886d8de3168f8d: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260806 09:33:38.688165 15716 log.cc:824] T 41fe54925c9a458183f7e5cf2649c40b P 61db43ea12324acfa22105bb59564a75: Log is configured to *not* fsync() on all Append() calls
I20260806 09:33:38.689335 15718 tablet_bootstrap.cc:654] T 41fe54925c9a458183f7e5cf2649c40b P c5ae52971ee4440ebf20daf5b3223d59: Neither blocks nor log segments found. Creating new log.
I20260806 09:33:38.690764 15718 log.cc:824] T 41fe54925c9a458183f7e5cf2649c40b P c5ae52971ee4440ebf20daf5b3223d59: Log is configured to *not* fsync() on all Append() calls
I20260806 09:33:38.692052 15716 tablet_bootstrap.cc:492] T 41fe54925c9a458183f7e5cf2649c40b P 61db43ea12324acfa22105bb59564a75: No bootstrap required, opened a new log
I20260806 09:33:38.692428 15716 ts_tablet_manager.cc:1397] T 41fe54925c9a458183f7e5cf2649c40b P 61db43ea12324acfa22105bb59564a75: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260806 09:33:38.694097 15718 tablet_bootstrap.cc:492] T 41fe54925c9a458183f7e5cf2649c40b P c5ae52971ee4440ebf20daf5b3223d59: No bootstrap required, opened a new log
I20260806 09:33:38.694370 15718 ts_tablet_manager.cc:1397] T 41fe54925c9a458183f7e5cf2649c40b P c5ae52971ee4440ebf20daf5b3223d59: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260806 09:33:38.695909 15715 raft_consensus.cc:348] T 41fe54925c9a458183f7e5cf2649c40b P 5f395e3ef35d4d0f99886d8de3168f8d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f395e3ef35d4d0f99886d8de3168f8d" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 42985 } } peers { permanent_uuid: "61db43ea12324acfa22105bb59564a75" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 41215 } } peers { permanent_uuid: "c5ae52971ee4440ebf20daf5b3223d59" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 33367 } }
I20260806 09:33:38.696290 15715 raft_consensus.cc:374] T 41fe54925c9a458183f7e5cf2649c40b P 5f395e3ef35d4d0f99886d8de3168f8d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:33:38.696468 15715 raft_consensus.cc:729] T 41fe54925c9a458183f7e5cf2649c40b P 5f395e3ef35d4d0f99886d8de3168f8d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f395e3ef35d4d0f99886d8de3168f8d, State: Initialized, Role: FOLLOWER
I20260806 09:33:38.697127 15715 consensus_queue.cc:260] T 41fe54925c9a458183f7e5cf2649c40b P 5f395e3ef35d4d0f99886d8de3168f8d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f395e3ef35d4d0f99886d8de3168f8d" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 42985 } } peers { permanent_uuid: "61db43ea12324acfa22105bb59564a75" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 41215 } } peers { permanent_uuid: "c5ae52971ee4440ebf20daf5b3223d59" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 33367 } }
I20260806 09:33:38.698284 15499 heartbeater.cc:503] Master 127.20.113.62:45571 was elected leader, sending a full tablet report...
I20260806 09:33:38.699010 15715 ts_tablet_manager.cc:1428] T 41fe54925c9a458183f7e5cf2649c40b P 5f395e3ef35d4d0f99886d8de3168f8d: Time spent starting tablet: real 0.012s user 0.011s sys 0.000s
I20260806 09:33:38.703581 15716 raft_consensus.cc:348] T 41fe54925c9a458183f7e5cf2649c40b P 61db43ea12324acfa22105bb59564a75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f395e3ef35d4d0f99886d8de3168f8d" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 42985 } } peers { permanent_uuid: "61db43ea12324acfa22105bb59564a75" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 41215 } } peers { permanent_uuid: "c5ae52971ee4440ebf20daf5b3223d59" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 33367 } }
I20260806 09:33:38.704139 15716 raft_consensus.cc:374] T 41fe54925c9a458183f7e5cf2649c40b P 61db43ea12324acfa22105bb59564a75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:33:38.704249 15716 raft_consensus.cc:729] T 41fe54925c9a458183f7e5cf2649c40b P 61db43ea12324acfa22105bb59564a75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61db43ea12324acfa22105bb59564a75, State: Initialized, Role: FOLLOWER
I20260806 09:33:38.704962 15716 consensus_queue.cc:260] T 41fe54925c9a458183f7e5cf2649c40b P 61db43ea12324acfa22105bb59564a75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f395e3ef35d4d0f99886d8de3168f8d" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 42985 } } peers { permanent_uuid: "61db43ea12324acfa22105bb59564a75" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 41215 } } peers { permanent_uuid: "c5ae52971ee4440ebf20daf5b3223d59" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 33367 } }
I20260806 09:33:38.705513 15718 raft_consensus.cc:348] T 41fe54925c9a458183f7e5cf2649c40b P c5ae52971ee4440ebf20daf5b3223d59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f395e3ef35d4d0f99886d8de3168f8d" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 42985 } } peers { permanent_uuid: "61db43ea12324acfa22105bb59564a75" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 41215 } } peers { permanent_uuid: "c5ae52971ee4440ebf20daf5b3223d59" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 33367 } }
I20260806 09:33:38.706001 15718 raft_consensus.cc:374] T 41fe54925c9a458183f7e5cf2649c40b P c5ae52971ee4440ebf20daf5b3223d59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:33:38.706223 15718 raft_consensus.cc:729] T 41fe54925c9a458183f7e5cf2649c40b P c5ae52971ee4440ebf20daf5b3223d59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5ae52971ee4440ebf20daf5b3223d59, State: Initialized, Role: FOLLOWER
I20260806 09:33:38.706985 15716 ts_tablet_manager.cc:1428] T 41fe54925c9a458183f7e5cf2649c40b P 61db43ea12324acfa22105bb59564a75: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260806 09:33:38.706864 15718 consensus_queue.cc:260] T 41fe54925c9a458183f7e5cf2649c40b P c5ae52971ee4440ebf20daf5b3223d59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f395e3ef35d4d0f99886d8de3168f8d" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 42985 } } peers { permanent_uuid: "61db43ea12324acfa22105bb59564a75" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 41215 } } peers { permanent_uuid: "c5ae52971ee4440ebf20daf5b3223d59" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 33367 } }
I20260806 09:33:38.708218 15690 heartbeater.cc:503] Master 127.20.113.62:45571 was elected leader, sending a full tablet report...
I20260806 09:33:38.708825 15718 ts_tablet_manager.cc:1428] T 41fe54925c9a458183f7e5cf2649c40b P c5ae52971ee4440ebf20daf5b3223d59: Time spent starting tablet: real 0.014s user 0.016s sys 0.000s
W20260806 09:33:38.734750 15500 tablet.cc:2369] T 41fe54925c9a458183f7e5cf2649c40b P 5f395e3ef35d4d0f99886d8de3168f8d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 09:33:38.815881 15691 tablet.cc:2369] T 41fe54925c9a458183f7e5cf2649c40b P c5ae52971ee4440ebf20daf5b3223d59: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 09:33:38.917004 15276 tablet.cc:2369] T 41fe54925c9a458183f7e5cf2649c40b P 61db43ea12324acfa22105bb59564a75: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 09:33:39.006897 15725 raft_consensus.cc:482] T 41fe54925c9a458183f7e5cf2649c40b P c5ae52971ee4440ebf20daf5b3223d59 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 09:33:39.007215 15725 raft_consensus.cc:504] T 41fe54925c9a458183f7e5cf2649c40b P c5ae52971ee4440ebf20daf5b3223d59 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f395e3ef35d4d0f99886d8de3168f8d" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 42985 } } peers { permanent_uuid: "61db43ea12324acfa22105bb59564a75" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 41215 } } peers { permanent_uuid: "c5ae52971ee4440ebf20daf5b3223d59" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 33367 } }
I20260806 09:33:39.009104 15725 leader_election.cc:288] T 41fe54925c9a458183f7e5cf2649c40b P c5ae52971ee4440ebf20daf5b3223d59 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5f395e3ef35d4d0f99886d8de3168f8d (127.20.113.2:42985), 61db43ea12324acfa22105bb59564a75 (127.20.113.1:41215)
I20260806 09:33:39.015946 15693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:39.009469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.2:42985 (local address 127.20.113.3:51967)
0806 09:33:39.009715 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:39.009738 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 09:33:39.009942 (+ 204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:33:39.010382 (+ 440us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:33:39.010392 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:33:39.010424 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:33:39.010642 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:33:39.010653 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:33:39.011940 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:33:39.011944 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:33:39.014001 (+ 2057us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:33:39.014017 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:33:39.015380 (+ 1363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:33:39.015385 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:33:39.015520 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:39.015544 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 09:33:39.015618 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 09:33:39.015716 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66}
I20260806 09:33:39.016088 15709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:39.009516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.3:51967 (local address 127.20.113.2:42985)
0806 09:33:39.009729 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0806 09:33:39.009737 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 09:33:39.009882 (+ 145us) server_negotiation.cc:407] Connection header received
0806 09:33:39.010112 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:33:39.010119 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:33:39.010223 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:33:39.010333 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:33:39.010799 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:39.011788 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:39.014160 (+ 2372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:39.015228 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:39.015386 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:39.015463 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 09:33:39.015673 (+ 210us) server_negotiation.cc:299] Negotiation successful
0806 09:33:39.015805 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 09:33:39.016763 15714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:39.010052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.3:47609 (local address 127.20.113.1:41215)
0806 09:33:39.010274 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0806 09:33:39.010281 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:33:39.010659 (+ 378us) server_negotiation.cc:407] Connection header received
0806 09:33:39.010812 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:33:39.010816 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:33:39.010901 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:33:39.011033 (+ 132us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:33:39.011570 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:39.012460 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:39.014628 (+ 2168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:39.015864 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:39.016026 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:39.016100 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 09:33:39.016444 (+ 344us) server_negotiation.cc:299] Negotiation successful
0806 09:33:39.016593 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 09:33:39.016913 15726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:39.009880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.1:41215 (local address 127.20.113.3:47609)
0806 09:33:39.010540 (+ 660us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:39.010558 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 09:33:39.010682 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:33:39.011077 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:33:39.011082 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:33:39.011106 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:33:39.011401 (+ 295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:33:39.011410 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:33:39.012620 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:33:39.012627 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:33:39.014475 (+ 1848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:33:39.014487 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:33:39.016087 (+ 1600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:33:39.016094 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:33:39.016247 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:39.016281 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 09:33:39.016404 (+ 123us) client_negotiation.cc:240] Negotiation successful
0806 09:33:39.016554 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":557,"thread_start_us":392,"threads_started":1}
I20260806 09:33:39.017452 15421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41fe54925c9a458183f7e5cf2649c40b" candidate_uuid: "c5ae52971ee4440ebf20daf5b3223d59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f395e3ef35d4d0f99886d8de3168f8d" is_pre_election: true
I20260806 09:33:39.017988 15421 raft_consensus.cc:2454] T 41fe54925c9a458183f7e5cf2649c40b P 5f395e3ef35d4d0f99886d8de3168f8d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c5ae52971ee4440ebf20daf5b3223d59 in term 0.
I20260806 09:33:39.017939 15201 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41fe54925c9a458183f7e5cf2649c40b" candidate_uuid: "c5ae52971ee4440ebf20daf5b3223d59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61db43ea12324acfa22105bb59564a75" is_pre_election: true
I20260806 09:33:39.018508 15201 raft_consensus.cc:2454] T 41fe54925c9a458183f7e5cf2649c40b P 61db43ea12324acfa22105bb59564a75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c5ae52971ee4440ebf20daf5b3223d59 in term 0.
I20260806 09:33:39.018708 15532 leader_election.cc:302] T 41fe54925c9a458183f7e5cf2649c40b P c5ae52971ee4440ebf20daf5b3223d59 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5f395e3ef35d4d0f99886d8de3168f8d, c5ae52971ee4440ebf20daf5b3223d59; no voters:
I20260806 09:33:39.019107 15725 raft_consensus.cc:2781] T 41fe54925c9a458183f7e5cf2649c40b P c5ae52971ee4440ebf20daf5b3223d59 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 09:33:39.019285 15725 raft_consensus.cc:482] T 41fe54925c9a458183f7e5cf2649c40b P c5ae52971ee4440ebf20daf5b3223d59 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 09:33:39.019407 15725 raft_consensus.cc:3037] T 41fe54925c9a458183f7e5cf2649c40b P c5ae52971ee4440ebf20daf5b3223d59 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:33:39.021699 15725 raft_consensus.cc:504] T 41fe54925c9a458183f7e5cf2649c40b P c5ae52971ee4440ebf20daf5b3223d59 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f395e3ef35d4d0f99886d8de3168f8d" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 42985 } } peers { permanent_uuid: "61db43ea12324acfa22105bb59564a75" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 41215 } } peers { permanent_uuid: "c5ae52971ee4440ebf20daf5b3223d59" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 33367 } }
I20260806 09:33:39.022292 15725 leader_election.cc:288] T 41fe54925c9a458183f7e5cf2649c40b P c5ae52971ee4440ebf20daf5b3223d59 [CANDIDATE]: Term 1 election: Requested vote from peers 5f395e3ef35d4d0f99886d8de3168f8d (127.20.113.2:42985), 61db43ea12324acfa22105bb59564a75 (127.20.113.1:41215)
I20260806 09:33:39.022706 15421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41fe54925c9a458183f7e5cf2649c40b" candidate_uuid: "c5ae52971ee4440ebf20daf5b3223d59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f395e3ef35d4d0f99886d8de3168f8d"
I20260806 09:33:39.022800 15201 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41fe54925c9a458183f7e5cf2649c40b" candidate_uuid: "c5ae52971ee4440ebf20daf5b3223d59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61db43ea12324acfa22105bb59564a75"
I20260806 09:33:39.022921 15421 raft_consensus.cc:3037] T 41fe54925c9a458183f7e5cf2649c40b P 5f395e3ef35d4d0f99886d8de3168f8d [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:33:39.023037 15201 raft_consensus.cc:3037] T 41fe54925c9a458183f7e5cf2649c40b P 61db43ea12324acfa22105bb59564a75 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:33:39.025364 15421 raft_consensus.cc:2454] T 41fe54925c9a458183f7e5cf2649c40b P 5f395e3ef35d4d0f99886d8de3168f8d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c5ae52971ee4440ebf20daf5b3223d59 in term 1.
I20260806 09:33:39.025367 15201 raft_consensus.cc:2454] T 41fe54925c9a458183f7e5cf2649c40b P 61db43ea12324acfa22105bb59564a75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c5ae52971ee4440ebf20daf5b3223d59 in term 1.
I20260806 09:33:39.025803 15532 leader_election.cc:302] T 41fe54925c9a458183f7e5cf2649c40b P c5ae52971ee4440ebf20daf5b3223d59 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5f395e3ef35d4d0f99886d8de3168f8d, c5ae52971ee4440ebf20daf5b3223d59; no voters:
I20260806 09:33:39.026108 15725 raft_consensus.cc:2781] T 41fe54925c9a458183f7e5cf2649c40b P c5ae52971ee4440ebf20daf5b3223d59 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:33:39.026578 15725 raft_consensus.cc:686] T 41fe54925c9a458183f7e5cf2649c40b P c5ae52971ee4440ebf20daf5b3223d59 [term 1 LEADER]: Becoming Leader. State: Replica: c5ae52971ee4440ebf20daf5b3223d59, State: Running, Role: LEADER
I20260806 09:33:39.027194 15725 consensus_queue.cc:237] T 41fe54925c9a458183f7e5cf2649c40b P c5ae52971ee4440ebf20daf5b3223d59 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f395e3ef35d4d0f99886d8de3168f8d" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 42985 } } peers { permanent_uuid: "61db43ea12324acfa22105bb59564a75" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 41215 } } peers { permanent_uuid: "c5ae52971ee4440ebf20daf5b3223d59" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 33367 } }
I20260806 09:33:39.032004 14998 catalog_manager.cc:5495] T 41fe54925c9a458183f7e5cf2649c40b P c5ae52971ee4440ebf20daf5b3223d59 reported cstate change: term changed from 0 to 1, leader changed from <none> to c5ae52971ee4440ebf20daf5b3223d59 (127.20.113.3). New cstate: current_term: 1 leader_uuid: "c5ae52971ee4440ebf20daf5b3223d59" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f395e3ef35d4d0f99886d8de3168f8d" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 42985 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61db43ea12324acfa22105bb59564a75" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 41215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c5ae52971ee4440ebf20daf5b3223d59" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 33367 } health_report { overall_health: HEALTHY } } }
I20260806 09:33:39.106356 15712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:39.100415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52158 (local address 127.20.113.3:33367)
0806 09:33:39.100652 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0806 09:33:39.100658 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:33:39.100710 (+ 52us) server_negotiation.cc:407] Connection header received
0806 09:33:39.100984 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:33:39.100989 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:33:39.101069 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:33:39.101231 (+ 162us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 09:33:39.101655 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:39.102443 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:39.103889 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:39.104405 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:39.104534 (+ 129us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:39.105201 (+ 667us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 09:33:39.105692 (+ 491us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 09:33:39.105880 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0806 09:33:39.106080 (+ 200us) server_negotiation.cc:299] Negotiation successful
0806 09:33:39.106128 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 09:33:39.114641 15421 raft_consensus.cc:1261] T 41fe54925c9a458183f7e5cf2649c40b P 5f395e3ef35d4d0f99886d8de3168f8d [term 1 FOLLOWER]: Refusing update from remote peer c5ae52971ee4440ebf20daf5b3223d59: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 09:33:39.114660 15201 raft_consensus.cc:1261] T 41fe54925c9a458183f7e5cf2649c40b P 61db43ea12324acfa22105bb59564a75 [term 1 FOLLOWER]: Refusing update from remote peer c5ae52971ee4440ebf20daf5b3223d59: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 09:33:39.115664 15727 consensus_queue.cc:1035] T 41fe54925c9a458183f7e5cf2649c40b P c5ae52971ee4440ebf20daf5b3223d59 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f395e3ef35d4d0f99886d8de3168f8d" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 42985 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 09:33:39.116061 15725 consensus_queue.cc:1035] T 41fe54925c9a458183f7e5cf2649c40b P c5ae52971ee4440ebf20daf5b3223d59 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61db43ea12324acfa22105bb59564a75" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 41215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 09:33:39.137781 15733 mvcc.cc:205] Tried to move back new op lower bound from 7315492123082559488 to 7315492122740133888. Current Snapshot: MvccSnapshot[applied={T|T < 7315492123082559488}]
I20260806 09:33:39.143586 14998 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34972:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 09:33:39.143906 14998 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34972:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 09:33:39.147575 14998 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 479b89c2a4a24353bd05e9981e0e6548: Sending DeleteTablet for 3 replicas of tablet 41fe54925c9a458183f7e5cf2649c40b
I20260806 09:33:39.148653 15390 tablet_service.cc:1514] Processing DeleteTablet for tablet 41fe54925c9a458183f7e5cf2649c40b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:33:39 UTC) from {username='kudu', principal='kudu/127.20.113.62@KRBTEST.COM'} at 127.0.0.1:60112
I20260806 09:33:39.148761 15597 tablet_service.cc:1514] Processing DeleteTablet for tablet 41fe54925c9a458183f7e5cf2649c40b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:33:39 UTC) from {username='kudu', principal='kudu/127.20.113.62@KRBTEST.COM'} at 127.0.0.1:52152
I20260806 09:33:39.148748 15172 tablet_service.cc:1514] Processing DeleteTablet for tablet 41fe54925c9a458183f7e5cf2649c40b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:33:39 UTC) from {username='kudu', principal='kudu/127.20.113.62@KRBTEST.COM'} at 127.0.0.1:38558
I20260806 09:33:39.151146 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 15083
I20260806 09:33:39.163817 15747 tablet_replica.cc:330] T 41fe54925c9a458183f7e5cf2649c40b P c5ae52971ee4440ebf20daf5b3223d59: stopping tablet replica
I20260806 09:33:39.164433 15747 raft_consensus.cc:2229] T 41fe54925c9a458183f7e5cf2649c40b P c5ae52971ee4440ebf20daf5b3223d59 [term 1 LEADER]: Raft consensus shutting down.
I20260806 09:33:39.168862 15744 tablet_replica.cc:330] T 41fe54925c9a458183f7e5cf2649c40b P 61db43ea12324acfa22105bb59564a75: stopping tablet replica
I20260806 09:33:39.169266 15744 raft_consensus.cc:2229] T 41fe54925c9a458183f7e5cf2649c40b P 61db43ea12324acfa22105bb59564a75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 09:33:39.169744 15744 raft_consensus.cc:2258] T 41fe54925c9a458183f7e5cf2649c40b P 61db43ea12324acfa22105bb59564a75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 09:33:39.169371 15747 raft_consensus.cc:2258] T 41fe54925c9a458183f7e5cf2649c40b P c5ae52971ee4440ebf20daf5b3223d59 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 09:33:39.171384 15743 tablet_replica.cc:330] T 41fe54925c9a458183f7e5cf2649c40b P 5f395e3ef35d4d0f99886d8de3168f8d: stopping tablet replica
I20260806 09:33:39.171818 15743 raft_consensus.cc:2229] T 41fe54925c9a458183f7e5cf2649c40b P 5f395e3ef35d4d0f99886d8de3168f8d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 09:33:39.172580 15747 ts_tablet_manager.cc:1905] T 41fe54925c9a458183f7e5cf2649c40b P c5ae52971ee4440ebf20daf5b3223d59: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 09:33:39.172878 15743 raft_consensus.cc:2258] T 41fe54925c9a458183f7e5cf2649c40b P 5f395e3ef35d4d0f99886d8de3168f8d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 09:33:39.173415 15744 ts_tablet_manager.cc:1905] T 41fe54925c9a458183f7e5cf2649c40b P 61db43ea12324acfa22105bb59564a75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 09:33:39.174923 15743 ts_tablet_manager.cc:1905] T 41fe54925c9a458183f7e5cf2649c40b P 5f395e3ef35d4d0f99886d8de3168f8d: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 06 09:33:39 dist-test-slave-45dx krb5kdc[14935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008816, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.113.1@KRBTEST.COM
I20260806 09:33:39.180739 15744 ts_tablet_manager.cc:1918] T 41fe54925c9a458183f7e5cf2649c40b P 61db43ea12324acfa22105bb59564a75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 09:33:39.180909 15744 log.cc:1196] T 41fe54925c9a458183f7e5cf2649c40b P 61db43ea12324acfa22105bb59564a75: Deleting WAL directory at /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786008495763054-20932-0/minicluster-data/ts-0/wal/wals/41fe54925c9a458183f7e5cf2649c40b
I20260806 09:33:39.181497 15744 ts_tablet_manager.cc:1939] T 41fe54925c9a458183f7e5cf2649c40b P 61db43ea12324acfa22105bb59564a75: Deleting consensus metadata
I20260806 09:33:39.182621 15743 ts_tablet_manager.cc:1918] T 41fe54925c9a458183f7e5cf2649c40b P 5f395e3ef35d4d0f99886d8de3168f8d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 09:33:39.182705 15747 ts_tablet_manager.cc:1918] T 41fe54925c9a458183f7e5cf2649c40b P c5ae52971ee4440ebf20daf5b3223d59: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 09:33:39.182909 15747 log.cc:1196] T 41fe54925c9a458183f7e5cf2649c40b P c5ae52971ee4440ebf20daf5b3223d59: Deleting WAL directory at /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786008495763054-20932-0/minicluster-data/ts-2/wal/wals/41fe54925c9a458183f7e5cf2649c40b
I20260806 09:33:39.182909 15743 log.cc:1196] T 41fe54925c9a458183f7e5cf2649c40b P 5f395e3ef35d4d0f99886d8de3168f8d: Deleting WAL directory at /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786008495763054-20932-0/minicluster-data/ts-1/wal/wals/41fe54925c9a458183f7e5cf2649c40b
I20260806 09:33:39.183449 15747 ts_tablet_manager.cc:1939] T 41fe54925c9a458183f7e5cf2649c40b P c5ae52971ee4440ebf20daf5b3223d59: Deleting consensus metadata
I20260806 09:33:39.183581 14979 catalog_manager.cc:4841] TS 61db43ea12324acfa22105bb59564a75 (127.20.113.1:41215): tablet 41fe54925c9a458183f7e5cf2649c40b (table test-table [id=fa7454d334124c65ae7728a1bfb87502]) successfully deleted
I20260806 09:33:39.184087 15743 ts_tablet_manager.cc:1939] T 41fe54925c9a458183f7e5cf2649c40b P 5f395e3ef35d4d0f99886d8de3168f8d: Deleting consensus metadata
I20260806 09:33:39.185261 14982 catalog_manager.cc:4841] TS c5ae52971ee4440ebf20daf5b3223d59 (127.20.113.3:33367): tablet 41fe54925c9a458183f7e5cf2649c40b (table test-table [id=fa7454d334124c65ae7728a1bfb87502]) successfully deleted
I20260806 09:33:39.186146 14979 catalog_manager.cc:4841] TS 5f395e3ef35d4d0f99886d8de3168f8d (127.20.113.2:42985): tablet 41fe54925c9a458183f7e5cf2649c40b (table test-table [id=fa7454d334124c65ae7728a1bfb87502]) successfully deleted
I20260806 09:33:39.189965 15714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:39.152517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38568 (local address 127.20.113.1:41215)
0806 09:33:39.152690 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0806 09:33:39.152695 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:33:39.157832 (+ 5137us) server_negotiation.cc:407] Connection header received
0806 09:33:39.158865 (+ 1033us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:33:39.158873 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:33:39.158969 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:33:39.159054 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:33:39.165178 (+ 6124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:39.166672 (+ 1494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:39.170660 (+ 3988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:39.171300 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:39.171437 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:39.182566 (+ 11129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:33:39.182584 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:33:39.182601 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:33:39.182650 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:33:39.186450 (+ 3800us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:33:39.187480 (+ 1030us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:33:39.187488 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:33:39.187491 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:33:39.187660 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:33:39.188150 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:33:39.188155 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:33:39.188157 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:33:39.188652 (+ 495us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:33:39.189240 (+ 588us) server_negotiation.cc:1035] Waiting for connection context
0806 09:33:39.189384 (+ 144us) server_negotiation.cc:299] Negotiation successful
0806 09:33:39.189556 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 09:33:39.190676 15263 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:33:39.281258 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 15083
I20260806 09:33:39.309048 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 15282
Aug 06 09:33:39 dist-test-slave-45dx krb5kdc[14935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008816, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.113.2@KRBTEST.COM
I20260806 09:33:39.326107 15709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:39.309955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60124 (local address 127.20.113.2:42985)
0806 09:33:39.310148 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0806 09:33:39.310155 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:33:39.310224 (+ 69us) server_negotiation.cc:407] Connection header received
0806 09:33:39.310472 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:33:39.310484 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:33:39.310556 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:33:39.310681 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:33:39.312003 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:39.312796 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:39.314166 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:39.314572 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:39.314709 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:39.319654 (+ 4945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:33:39.319671 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:33:39.319687 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:33:39.319723 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:33:39.322822 (+ 3099us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:33:39.323611 (+ 789us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:33:39.323620 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:33:39.323622 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:33:39.323785 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:33:39.324225 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:33:39.324230 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:33:39.324233 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:33:39.324781 (+ 548us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:33:39.325162 (+ 381us) server_negotiation.cc:1035] Waiting for connection context
0806 09:33:39.325465 (+ 303us) server_negotiation.cc:299] Negotiation successful
0806 09:33:39.325730 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260806 09:33:39.326958 15488 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:33:39.418040 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 15282
I20260806 09:33:39.455188 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 15506
Aug 06 09:33:39 dist-test-slave-45dx krb5kdc[14935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008816, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.113.3@KRBTEST.COM
I20260806 09:33:39.472263 15712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:39.456083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52168 (local address 127.20.113.3:33367)
0806 09:33:39.456309 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0806 09:33:39.456349 (+ 40us) server_negotiation.cc:399] Waiting for connection header
0806 09:33:39.456371 (+ 22us) server_negotiation.cc:407] Connection header received
0806 09:33:39.456513 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:33:39.456518 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:33:39.456584 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:33:39.456699 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:33:39.458149 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:39.459031 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:39.460530 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:39.461013 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:39.461173 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:39.465962 (+ 4789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:33:39.465976 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:33:39.465993 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:33:39.466033 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:33:39.469284 (+ 3251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:33:39.470045 (+ 761us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:33:39.470052 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:33:39.470055 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:33:39.470210 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:33:39.470604 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:33:39.470607 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:33:39.470609 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:33:39.470963 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:33:39.471328 (+ 365us) server_negotiation.cc:1035] Waiting for connection context
0806 09:33:39.471647 (+ 319us) server_negotiation.cc:299] Negotiation successful
0806 09:33:39.471895 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260806 09:33:39.473285 15685 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:33:39.599911 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 15506
I20260806 09:33:39.628262 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 14954
I20260806 09:33:39.629518 15039 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:33:39.741466 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 14954
2026-08-06T09:33:39Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (5086 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-06T09:33:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T09:33:39Z Disabled control of system clock
I20260806 09:33:39.855177 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786008495763054-20932-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786008495763054-20932-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786008495763054-20932-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.113.62:37553
--webserver_interface=127.20.113.62
--webserver_port=0
--builtin_ntp_servers=127.20.113.20:37137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.113.62:37553
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 09:33:40.213085 15759 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:33:40.213363 15759 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:33:40.213442 15759 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:33:40.220360 15759 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 09:33:40.220451 15759 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:33:40.220486 15759 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:33:40.220546 15759 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 09:33:40.220584 15759 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 09:33:40.229254 15759 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:37137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786008495763054-20932-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786008495763054-20932-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.113.62:37553
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.113.62:37553
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.113.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786008495763054-20932-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:33:40.230464 15759 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:33:40.232249 15759 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:33:40.246744 15766 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:33:40.246804 15765 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:33:40.246846 15771 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:33:40.247717 15759 server_base.cc:1034] running on GCE node
I20260806 09:33:40.248590 15759 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:33:40.250077 15759 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:33:40.251305 15759 hybrid_clock.cc:648] HybridClock initialized: now 1786008820251257 us; error 44 us; skew 500 ppm
I20260806 09:33:40.251637 15759 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 09:33:40.253968 15759 webserver.cc:460] Webserver started at http://127.20.113.62:41457/ using document root <none> and password file <none>
I20260806 09:33:40.254680 15759 fs_manager.cc:359] Metadata directory not provided
I20260806 09:33:40.254812 15759 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:33:40.255146 15759 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:33:40.258095 15759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786008495763054-20932-0/minicluster-data/master-0/data/instance:
uuid: "884cc132a6374291a65ef69881c3cec4"
format_stamp: "Formatted at 2026-08-06 09:33:40 on dist-test-slave-45dx"
server_key: "5cb579b396133bfba9fa8fb42b68a7fd"
server_key_iv: "caf795bdc9ad7e160001f305c239b664"
server_key_version: "encryptionkey@0"
I20260806 09:33:40.258924 15759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786008495763054-20932-0/minicluster-data/master-0/wal/instance:
uuid: "884cc132a6374291a65ef69881c3cec4"
format_stamp: "Formatted at 2026-08-06 09:33:40 on dist-test-slave-45dx"
server_key: "5cb579b396133bfba9fa8fb42b68a7fd"
server_key_iv: "caf795bdc9ad7e160001f305c239b664"
server_key_version: "encryptionkey@0"
I20260806 09:33:40.265595 15759 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260806 09:33:40.269738 15779 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:33:40.271073 15759 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 09:33:40.271260 15759 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786008495763054-20932-0/minicluster-data/master-0/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786008495763054-20932-0/minicluster-data/master-0/wal
uuid: "884cc132a6374291a65ef69881c3cec4"
format_stamp: "Formatted at 2026-08-06 09:33:40 on dist-test-slave-45dx"
server_key: "5cb579b396133bfba9fa8fb42b68a7fd"
server_key_iv: "caf795bdc9ad7e160001f305c239b664"
server_key_version: "encryptionkey@0"
I20260806 09:33:40.271472 15759 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786008495763054-20932-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786008495763054-20932-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786008495763054-20932-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:33:40.290292 15759 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:33:40.291095 15759 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:33:40.291386 15759 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:33:40.309863 15759 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.62:37553
I20260806 09:33:40.309880 15862 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.62:37553 every 8 connection(s)
I20260806 09:33:40.311305 15759 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
I20260806 09:33:40.314882 15863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:33:40.319667 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 15759
I20260806 09:33:40.319856 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786008495763054-20932-0/minicluster-data/master-0/wal/instance
I20260806 09:33:40.320469 20932 external_mini_cluster.cc:1428] Setting key 769f5399bc3911d183d0a59e01428dd7
I20260806 09:33:40.327632 15863 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 884cc132a6374291a65ef69881c3cec4: Bootstrap starting.
I20260806 09:33:40.331503 15863 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 884cc132a6374291a65ef69881c3cec4: Neither blocks nor log segments found. Creating new log.
I20260806 09:33:40.332729 15863 log.cc:824] T 00000000000000000000000000000000 P 884cc132a6374291a65ef69881c3cec4: Log is configured to *not* fsync() on all Append() calls
I20260806 09:33:40.336395 15863 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 884cc132a6374291a65ef69881c3cec4: No bootstrap required, opened a new log
I20260806 09:33:40.337520 15866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:40.322695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36000 (local address 127.20.113.62:37553)
0806 09:33:40.324089 (+ 1394us) server_negotiation.cc:206] Beginning negotiation
0806 09:33:40.324103 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 09:33:40.324257 (+ 154us) server_negotiation.cc:407] Connection header received
0806 09:33:40.326869 (+ 2612us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:33:40.326896 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:33:40.327484 (+ 588us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:33:40.327919 (+ 435us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:33:40.328505 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:40.330538 (+ 2033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:40.332083 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:40.333151 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:40.333399 (+ 248us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:40.334607 (+ 1208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:33:40.334619 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:33:40.334637 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:33:40.335380 (+ 743us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:33:40.335490 (+ 110us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:33:40.335522 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:33:40.335641 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0806 09:33:40.335874 (+ 233us) server_negotiation.cc:299] Negotiation successful
0806 09:33:40.336195 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":842,"thread_start_us":513,"threads_started":1}
I20260806 09:33:40.344209 15863 raft_consensus.cc:348] T 00000000000000000000000000000000 P 884cc132a6374291a65ef69881c3cec4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "884cc132a6374291a65ef69881c3cec4" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 37553 } }
I20260806 09:33:40.344621 15863 raft_consensus.cc:374] T 00000000000000000000000000000000 P 884cc132a6374291a65ef69881c3cec4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:33:40.344703 15863 raft_consensus.cc:729] T 00000000000000000000000000000000 P 884cc132a6374291a65ef69881c3cec4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 884cc132a6374291a65ef69881c3cec4, State: Initialized, Role: FOLLOWER
I20260806 09:33:40.345180 15863 consensus_queue.cc:260] T 00000000000000000000000000000000 P 884cc132a6374291a65ef69881c3cec4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "884cc132a6374291a65ef69881c3cec4" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 37553 } }
I20260806 09:33:40.345386 15863 raft_consensus.cc:388] T 00000000000000000000000000000000 P 884cc132a6374291a65ef69881c3cec4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 09:33:40.345484 15863 raft_consensus.cc:482] T 00000000000000000000000000000000 P 884cc132a6374291a65ef69881c3cec4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 09:33:40.345607 15863 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 884cc132a6374291a65ef69881c3cec4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:33:40.347705 15863 raft_consensus.cc:504] T 00000000000000000000000000000000 P 884cc132a6374291a65ef69881c3cec4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "884cc132a6374291a65ef69881c3cec4" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 37553 } }
I20260806 09:33:40.348137 15863 leader_election.cc:302] T 00000000000000000000000000000000 P 884cc132a6374291a65ef69881c3cec4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 884cc132a6374291a65ef69881c3cec4; no voters:
I20260806 09:33:40.348727 15863 leader_election.cc:288] T 00000000000000000000000000000000 P 884cc132a6374291a65ef69881c3cec4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 09:33:40.348928 15868 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 884cc132a6374291a65ef69881c3cec4 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:33:40.349692 15868 raft_consensus.cc:686] T 00000000000000000000000000000000 P 884cc132a6374291a65ef69881c3cec4 [term 1 LEADER]: Becoming Leader. State: Replica: 884cc132a6374291a65ef69881c3cec4, State: Running, Role: LEADER
I20260806 09:33:40.350318 15868 consensus_queue.cc:237] T 00000000000000000000000000000000 P 884cc132a6374291a65ef69881c3cec4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "884cc132a6374291a65ef69881c3cec4" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 37553 } }
I20260806 09:33:40.351056 15863 sys_catalog.cc:564] T 00000000000000000000000000000000 P 884cc132a6374291a65ef69881c3cec4 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 09:33:40.353549 15871 sys_catalog.cc:455] T 00000000000000000000000000000000 P 884cc132a6374291a65ef69881c3cec4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "884cc132a6374291a65ef69881c3cec4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "884cc132a6374291a65ef69881c3cec4" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 37553 } } }
I20260806 09:33:40.353648 15873 sys_catalog.cc:455] T 00000000000000000000000000000000 P 884cc132a6374291a65ef69881c3cec4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 884cc132a6374291a65ef69881c3cec4. Latest consensus state: current_term: 1 leader_uuid: "884cc132a6374291a65ef69881c3cec4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "884cc132a6374291a65ef69881c3cec4" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 37553 } } }
I20260806 09:33:40.354060 15873 sys_catalog.cc:458] T 00000000000000000000000000000000 P 884cc132a6374291a65ef69881c3cec4 [sys.catalog]: This master's current role is: LEADER
I20260806 09:33:40.353947 15871 sys_catalog.cc:458] T 00000000000000000000000000000000 P 884cc132a6374291a65ef69881c3cec4 [sys.catalog]: This master's current role is: LEADER
I20260806 09:33:40.357481 15878 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 09:33:40.361861 15878 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 09:33:40.371184 15878 catalog_manager.cc:1347] Generated new cluster ID: 6db1dd5a6d5e408ba9ec2693e390f382
I20260806 09:33:40.371286 15878 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 09:33:40.377893 15878 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 09:33:40.379040 15878 catalog_manager.cc:1504] Loading token signing keys...
I20260806 09:33:40.397146 15878 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 884cc132a6374291a65ef69881c3cec4: Generated new TSK 0
I20260806 09:33:40.398079 15878 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 09:33:40.410295 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786008495763054-20932-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786008495763054-20932-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786008495763054-20932-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786008495763054-20932-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.113.1:0
--local_ip_for_outbound_sockets=127.20.113.1
--webserver_interface=127.20.113.1
--webserver_port=0
--tserver_master_addrs=127.20.113.62:37553
--builtin_ntp_servers=127.20.113.20:37137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 09:33:40.778898 15895 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:33:40.779155 15895 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:33:40.779235 15895 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:33:40.786253 15895 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:33:40.786345 15895 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:33:40.786451 15895 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.113.1
I20260806 09:33:40.794986 15895 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:37137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786008495763054-20932-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786008495763054-20932-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.113.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786008495763054-20932-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.113.1
--webserver_port=0
--tserver_master_addrs=127.20.113.62:37553
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.113.1
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786008495763054-20932-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:33:40.796180 15895 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:33:40.797907 15895 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:33:40.810351 15904 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:33:40.810315 15908 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:33:40.810446 15905 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:33:40.811446 15895 server_base.cc:1034] running on GCE node
I20260806 09:33:40.812052 15895 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:33:40.812956 15895 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:33:40.814163 15895 hybrid_clock.cc:648] HybridClock initialized: now 1786008820814109 us; error 51 us; skew 500 ppm
I20260806 09:33:40.814425 15895 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 09:33:40.816463 15895 webserver.cc:460] Webserver started at http://127.20.113.1:37995/ using document root <none> and password file <none>
I20260806 09:33:40.817081 15895 fs_manager.cc:359] Metadata directory not provided
I20260806 09:33:40.817183 15895 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:33:40.817427 15895 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:33:40.820050 15895 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786008495763054-20932-0/minicluster-data/ts-0/data/instance:
uuid: "6f7ceb74338248eaa57273407aaaab3f"
format_stamp: "Formatted at 2026-08-06 09:33:40 on dist-test-slave-45dx"
server_key: "bcf19adb1a1c0acfee16eafbbefc5d8d"
server_key_iv: "db125e029d1cf102aa639cc97dce2218"
server_key_version: "encryptionkey@0"
I20260806 09:33:40.820873 15895 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786008495763054-20932-0/minicluster-data/ts-0/wal/instance:
uuid: "6f7ceb74338248eaa57273407aaaab3f"
format_stamp: "Formatted at 2026-08-06 09:33:40 on dist-test-slave-45dx"
server_key: "bcf19adb1a1c0acfee16eafbbefc5d8d"
server_key_iv: "db125e029d1cf102aa639cc97dce2218"
server_key_version: "encryptionkey@0"
I20260806 09:33:40.825809 15895 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 09:33:40.829144 15918 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:33:40.830468 15895 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 09:33:40.830597 15895 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786008495763054-20932-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786008495763054-20932-0/minicluster-data/ts-0/wal
uuid: "6f7ceb74338248eaa57273407aaaab3f"
format_stamp: "Formatted at 2026-08-06 09:33:40 on dist-test-slave-45dx"
server_key: "bcf19adb1a1c0acfee16eafbbefc5d8d"
server_key_iv: "db125e029d1cf102aa639cc97dce2218"
server_key_version: "encryptionkey@0"
I20260806 09:33:40.830763 15895 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786008495763054-20932-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786008495763054-20932-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786008495763054-20932-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:33:40.846335 15895 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:33:40.847067 15895 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:33:40.847334 15895 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:33:40.848198 15895 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:33:40.849610 15895 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:33:40.849697 15895 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:33:40.849777 15895 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:33:40.849830 15895 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:33:40.883060 15895 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.1:39695
I20260806 09:33:40.883129 16075 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.1:39695 every 8 connection(s)
I20260806 09:33:40.884359 15895 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786008495763054-20932-0/minicluster-data/ts-0/data/info.pb
I20260806 09:33:40.886310 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 15895
I20260806 09:33:40.886495 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786008495763054-20932-0/minicluster-data/ts-0/wal/instance
I20260806 09:33:40.887013 20932 external_mini_cluster.cc:1428] Setting key 96dbb0f1303620e5c43cc0d194d677a7
I20260806 09:33:40.890332 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786008495763054-20932-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786008495763054-20932-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786008495763054-20932-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786008495763054-20932-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.113.2:0
--local_ip_for_outbound_sockets=127.20.113.2
--webserver_interface=127.20.113.2
--webserver_port=0
--tserver_master_addrs=127.20.113.62:37553
--builtin_ntp_servers=127.20.113.20:37137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 09:33:40.899118 16082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:40.886919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.1:54493 (local address 127.20.113.62:37553)
0806 09:33:40.887490 (+ 571us) server_negotiation.cc:206] Beginning negotiation
0806 09:33:40.887497 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:33:40.888803 (+ 1306us) server_negotiation.cc:407] Connection header received
0806 09:33:40.891326 (+ 2523us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:33:40.891331 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:33:40.891410 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:33:40.891498 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:33:40.892838 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:40.893542 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:40.895892 (+ 2350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:40.896645 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:40.896779 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:40.898129 (+ 1350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:33:40.898137 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:33:40.898144 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:33:40.898170 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:33:40.898223 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:33:40.898241 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:33:40.898364 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0806 09:33:40.898567 (+ 203us) server_negotiation.cc:299] Negotiation successful
0806 09:33:40.898675 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":440,"thread_start_us":310,"threads_started":1}
I20260806 09:33:40.899843 16083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:40.887562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:37553 (local address 127.20.113.1:54493)
0806 09:33:40.888661 (+ 1099us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:40.888695 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 09:33:40.891047 (+ 2352us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:33:40.891749 (+ 702us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:33:40.891762 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:33:40.891866 (+ 104us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:33:40.892644 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:33:40.892669 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:33:40.893715 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:33:40.893723 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:33:40.895716 (+ 1993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:33:40.895734 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:33:40.896871 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:33:40.896877 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:33:40.897066 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:40.897812 (+ 746us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 09:33:40.897815 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:33:40.897866 (+ 51us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 09:33:40.897937 (+ 71us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:33:40.898361 (+ 424us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:33:40.898369 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:33:40.898375 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 09:33:40.898507 (+ 132us) client_negotiation.cc:240] Negotiation successful
0806 09:33:40.898792 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":623,"thread_start_us":371,"threads_started":1}
I20260806 09:33:40.901275 16078 heartbeater.cc:347] Connected to a master server at 127.20.113.62:37553
I20260806 09:33:40.901574 16078 heartbeater.cc:464] Registering TS with master...
I20260806 09:33:40.902456 16078 heartbeater.cc:511] Master 127.20.113.62:37553 requested a full tablet report, sending...
I20260806 09:33:40.904614 15801 ts_manager.cc:194] Registered new tserver with Master: 6f7ceb74338248eaa57273407aaaab3f (127.20.113.1:39695)
I20260806 09:33:40.906006 15801 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.113.1:54493
W20260806 09:33:41.233965 16084 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:33:41.234251 16084 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:33:41.234344 16084 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:33:41.241312 16084 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:33:41.241405 16084 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:33:41.241505 16084 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.113.2
I20260806 09:33:41.250151 16084 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:37137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786008495763054-20932-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786008495763054-20932-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.113.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786008495763054-20932-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.113.2
--webserver_port=0
--tserver_master_addrs=127.20.113.62:37553
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.113.2
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786008495763054-20932-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:33:41.251430 16084 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:33:41.253260 16084 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:33:41.266219 16091 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:33:41.266254 16092 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:33:41.266184 16094 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:33:41.267319 16084 server_base.cc:1034] running on GCE node
I20260806 09:33:41.267963 16084 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:33:41.268947 16084 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:33:41.270179 16084 hybrid_clock.cc:648] HybridClock initialized: now 1786008821270142 us; error 51 us; skew 500 ppm
I20260806 09:33:41.270484 16084 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 09:33:41.272835 16084 webserver.cc:460] Webserver started at http://127.20.113.2:46665/ using document root <none> and password file <none>
I20260806 09:33:41.273468 16084 fs_manager.cc:359] Metadata directory not provided
I20260806 09:33:41.273617 16084 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:33:41.273952 16084 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:33:41.276964 16084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786008495763054-20932-0/minicluster-data/ts-1/data/instance:
uuid: "bdbf625fbcda46ec9a5f9a0faadaa31c"
format_stamp: "Formatted at 2026-08-06 09:33:41 on dist-test-slave-45dx"
server_key: "a8de9fd95278187d4da6cadc6168e867"
server_key_iv: "527fd64e25a33e50eda6581237f78c75"
server_key_version: "encryptionkey@0"
I20260806 09:33:41.277832 16084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786008495763054-20932-0/minicluster-data/ts-1/wal/instance:
uuid: "bdbf625fbcda46ec9a5f9a0faadaa31c"
format_stamp: "Formatted at 2026-08-06 09:33:41 on dist-test-slave-45dx"
server_key: "a8de9fd95278187d4da6cadc6168e867"
server_key_iv: "527fd64e25a33e50eda6581237f78c75"
server_key_version: "encryptionkey@0"
I20260806 09:33:41.282722 16084 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260806 09:33:41.286085 16102 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:33:41.287452 16084 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 09:33:41.287614 16084 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786008495763054-20932-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786008495763054-20932-0/minicluster-data/ts-1/wal
uuid: "bdbf625fbcda46ec9a5f9a0faadaa31c"
format_stamp: "Formatted at 2026-08-06 09:33:41 on dist-test-slave-45dx"
server_key: "a8de9fd95278187d4da6cadc6168e867"
server_key_iv: "527fd64e25a33e50eda6581237f78c75"
server_key_version: "encryptionkey@0"
I20260806 09:33:41.287803 16084 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786008495763054-20932-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786008495763054-20932-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786008495763054-20932-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:33:41.305462 16084 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:33:41.306172 16084 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:33:41.306425 16084 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:33:41.307310 16084 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:33:41.308759 16084 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:33:41.308847 16084 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:33:41.308919 16084 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:33:41.308995 16084 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:33:41.352763 16084 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.2:33525
I20260806 09:33:41.352818 16264 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.2:33525 every 8 connection(s)
I20260806 09:33:41.354166 16084 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786008495763054-20932-0/minicluster-data/ts-1/data/info.pb
I20260806 09:33:41.356537 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 16084
I20260806 09:33:41.356750 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786008495763054-20932-0/minicluster-data/ts-1/wal/instance
I20260806 09:33:41.357316 20932 external_mini_cluster.cc:1428] Setting key 82f4b5f378523257678ce0f64b42c24d
I20260806 09:33:41.360530 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786008495763054-20932-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786008495763054-20932-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786008495763054-20932-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786008495763054-20932-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.113.3:0
--local_ip_for_outbound_sockets=127.20.113.3
--webserver_interface=127.20.113.3
--webserver_port=0
--tserver_master_addrs=127.20.113.62:37553
--builtin_ntp_servers=127.20.113.20:37137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 09:33:41.369021 16082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:41.356925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.2:55837 (local address 127.20.113.62:37553)
0806 09:33:41.357113 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0806 09:33:41.357117 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 09:33:41.358924 (+ 1807us) server_negotiation.cc:407] Connection header received
0806 09:33:41.361400 (+ 2476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:33:41.361404 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:33:41.361466 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:33:41.361546 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:33:41.363064 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:41.363782 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:41.366204 (+ 2422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:41.366574 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:41.366667 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:41.368281 (+ 1614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:33:41.368285 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:33:41.368288 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:33:41.368370 (+ 82us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:33:41.368402 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:33:41.368415 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:33:41.368484 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 09:33:41.368731 (+ 247us) server_negotiation.cc:299] Negotiation successful
0806 09:33:41.368806 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 09:33:41.370390 16267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:41.357552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:37553 (local address 127.20.113.2:55837)
0806 09:33:41.358753 (+ 1201us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:41.358808 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 09:33:41.361119 (+ 2311us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:33:41.361895 (+ 776us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:33:41.361908 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:33:41.362029 (+ 121us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:33:41.362873 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:33:41.362902 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:33:41.363949 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:33:41.363957 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:33:41.366064 (+ 2107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:33:41.366082 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:33:41.366799 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:33:41.366809 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:33:41.367128 (+ 319us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:41.367922 (+ 794us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 09:33:41.367928 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:33:41.367980 (+ 52us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 09:33:41.368090 (+ 110us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:33:41.368510 (+ 420us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:33:41.368523 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:33:41.368533 (+ 10us) client_negotiation.cc:769] Sending connection context
0806 09:33:41.368718 (+ 185us) client_negotiation.cc:240] Negotiation successful
0806 09:33:41.369119 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":663,"thread_start_us":439,"threads_started":1}
I20260806 09:33:41.371797 16265 heartbeater.cc:347] Connected to a master server at 127.20.113.62:37553
I20260806 09:33:41.372102 16265 heartbeater.cc:464] Registering TS with master...
I20260806 09:33:41.372841 16265 heartbeater.cc:511] Master 127.20.113.62:37553 requested a full tablet report, sending...
I20260806 09:33:41.374181 15801 ts_manager.cc:194] Registered new tserver with Master: bdbf625fbcda46ec9a5f9a0faadaa31c (127.20.113.2:33525)
I20260806 09:33:41.375012 15801 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.113.2:55837
W20260806 09:33:41.710963 16268 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:33:41.711318 16268 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:33:41.711472 16268 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:33:41.718833 16268 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:33:41.718968 16268 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:33:41.719141 16268 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.113.3
I20260806 09:33:41.728123 16268 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:37137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786008495763054-20932-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786008495763054-20932-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.113.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786008495763054-20932-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.113.3
--webserver_port=0
--tserver_master_addrs=127.20.113.62:37553
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.113.3
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786008495763054-20932-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:33:41.729506 16268 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:33:41.731364 16268 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:33:41.744800 16274 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:33:41.744913 16278 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:33:41.744762 16276 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:33:41.745945 16268 server_base.cc:1034] running on GCE node
I20260806 09:33:41.746573 16268 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:33:41.747526 16268 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:33:41.748764 16268 hybrid_clock.cc:648] HybridClock initialized: now 1786008821748709 us; error 62 us; skew 500 ppm
I20260806 09:33:41.749094 16268 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 09:33:41.751605 16268 webserver.cc:460] Webserver started at http://127.20.113.3:41325/ using document root <none> and password file <none>
I20260806 09:33:41.752424 16268 fs_manager.cc:359] Metadata directory not provided
I20260806 09:33:41.752561 16268 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:33:41.752897 16268 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:33:41.755820 16268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786008495763054-20932-0/minicluster-data/ts-2/data/instance:
uuid: "72cb1184a446423db895b3bf2109231e"
format_stamp: "Formatted at 2026-08-06 09:33:41 on dist-test-slave-45dx"
server_key: "a60bed2538b62627f3615ec68d5db96e"
server_key_iv: "eac188da2fe24fae8b8905da2040e5b5"
server_key_version: "encryptionkey@0"
I20260806 09:33:41.756767 16268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786008495763054-20932-0/minicluster-data/ts-2/wal/instance:
uuid: "72cb1184a446423db895b3bf2109231e"
format_stamp: "Formatted at 2026-08-06 09:33:41 on dist-test-slave-45dx"
server_key: "a60bed2538b62627f3615ec68d5db96e"
server_key_iv: "eac188da2fe24fae8b8905da2040e5b5"
server_key_version: "encryptionkey@0"
I20260806 09:33:41.761809 16268 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 09:33:41.765276 16289 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:33:41.766688 16268 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 09:33:41.766870 16268 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786008495763054-20932-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786008495763054-20932-0/minicluster-data/ts-2/wal
uuid: "72cb1184a446423db895b3bf2109231e"
format_stamp: "Formatted at 2026-08-06 09:33:41 on dist-test-slave-45dx"
server_key: "a60bed2538b62627f3615ec68d5db96e"
server_key_iv: "eac188da2fe24fae8b8905da2040e5b5"
server_key_version: "encryptionkey@0"
I20260806 09:33:41.767083 16268 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786008495763054-20932-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786008495763054-20932-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786008495763054-20932-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:33:41.780458 16268 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:33:41.781324 16268 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:33:41.781651 16268 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:33:41.782675 16268 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:33:41.784266 16268 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:33:41.784444 16268 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260806 09:33:41.784586 16268 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:33:41.784687 16268 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:33:41.819521 16268 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.3:40437
I20260806 09:33:41.819609 16456 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.3:40437 every 8 connection(s)
I20260806 09:33:41.820986 16268 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786008495763054-20932-0/minicluster-data/ts-2/data/info.pb
I20260806 09:33:41.826979 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 16268
I20260806 09:33:41.827162 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786008495763054-20932-0/minicluster-data/ts-2/wal/instance
I20260806 09:33:41.827636 20932 external_mini_cluster.cc:1428] Setting key 8c21c70f129c0c0dd94b74eca7779344
I20260806 09:33:41.833796 16082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:41.823621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.3:57385 (local address 127.20.113.62:37553)
0806 09:33:41.823798 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0806 09:33:41.823802 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 09:33:41.825129 (+ 1327us) server_negotiation.cc:407] Connection header received
0806 09:33:41.826826 (+ 1697us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:33:41.826832 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:33:41.826922 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:33:41.827030 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:33:41.828245 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:41.829201 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:41.831100 (+ 1899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:41.831501 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:41.831636 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:41.832966 (+ 1330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:33:41.832973 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:33:41.832977 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:33:41.833001 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:33:41.833030 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:33:41.833042 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:33:41.833138 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0806 09:33:41.833363 (+ 225us) server_negotiation.cc:299] Negotiation successful
0806 09:33:41.833459 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 09:33:41.834865 16462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:41.824046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:37553 (local address 127.20.113.3:57385)
0806 09:33:41.824960 (+ 914us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:41.825002 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 09:33:41.826569 (+ 1567us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:33:41.827320 (+ 751us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:33:41.827334 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:33:41.827430 (+ 96us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:33:41.828072 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:33:41.828089 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:33:41.829372 (+ 1283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:33:41.829377 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:33:41.830945 (+ 1568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:33:41.830958 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:33:41.831725 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:33:41.831732 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:33:41.831957 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:41.832643 (+ 686us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 09:33:41.832645 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:33:41.832682 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 09:33:41.832750 (+ 68us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:33:41.833139 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:33:41.833153 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:33:41.833162 (+ 9us) client_negotiation.cc:769] Sending connection context
0806 09:33:41.833345 (+ 183us) client_negotiation.cc:240] Negotiation successful
0806 09:33:41.833703 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":320,"threads_started":1}
I20260806 09:33:41.836102 16458 heartbeater.cc:347] Connected to a master server at 127.20.113.62:37553
I20260806 09:33:41.836433 16458 heartbeater.cc:464] Registering TS with master...
I20260806 09:33:41.837126 16458 heartbeater.cc:511] Master 127.20.113.62:37553 requested a full tablet report, sending...
I20260806 09:33:41.838560 15801 ts_manager.cc:194] Registered new tserver with Master: 72cb1184a446423db895b3bf2109231e (127.20.113.3:40437)
I20260806 09:33:41.839339 15801 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.113.3:57385
I20260806 09:33:41.842846 20932 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 09:33:41.852918 16082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:41.846916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36008 (local address 127.20.113.62:37553)
0806 09:33:41.847113 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0806 09:33:41.847121 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 09:33:41.847575 (+ 454us) server_negotiation.cc:407] Connection header received
0806 09:33:41.847694 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:33:41.847697 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:33:41.847764 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:33:41.847850 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:33:41.848291 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:41.849011 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:41.850386 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:41.850864 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:41.850955 (+ 91us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:41.852121 (+ 1166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:33:41.852126 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:33:41.852129 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:33:41.852144 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:33:41.852171 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:33:41.852182 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:33:41.852344 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0806 09:33:41.852555 (+ 211us) server_negotiation.cc:299] Negotiation successful
0806 09:33:41.852625 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 09:33:41.856853 15801 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36008:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 09:33:41.864037 15801 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 09:33:41.898389 16473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:41.875802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.2:33525 (local address 127.0.0.1:44740)
0806 09:33:41.885990 (+ 10188us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:41.886037 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 09:33:41.886227 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:33:41.890016 (+ 3789us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:33:41.890024 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:33:41.890055 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:33:41.890499 (+ 444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:33:41.890511 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:33:41.893369 (+ 2858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:33:41.893374 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:33:41.895657 (+ 2283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:33:41.895669 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:33:41.897232 (+ 1563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:33:41.897240 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:33:41.897518 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:41.897644 (+ 126us) client_negotiation.cc:769] Sending connection context
0806 09:33:41.897789 (+ 145us) client_negotiation.cc:240] Negotiation successful
0806 09:33:41.897992 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9920,"mutex_wait_us":33,"thread_start_us":17069,"threads_started":1}
I20260806 09:33:41.899201 16474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:41.876027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44740 (local address 127.20.113.2:33525)
0806 09:33:41.889164 (+ 13137us) server_negotiation.cc:206] Beginning negotiation
0806 09:33:41.889172 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 09:33:41.889201 (+ 29us) server_negotiation.cc:407] Connection header received
0806 09:33:41.889294 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:33:41.889302 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:33:41.889568 (+ 266us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:33:41.889743 (+ 175us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:33:41.890721 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:41.892904 (+ 2183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:41.895844 (+ 2940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:41.897016 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:41.897166 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:41.897456 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0806 09:33:41.898661 (+ 1205us) server_negotiation.cc:299] Negotiation successful
0806 09:33:41.898848 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12978,"thread_start_us":14608,"threads_started":1}
I20260806 09:33:41.901837 16472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:41.876380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.1:39695 (local address 127.0.0.1:54012)
0806 09:33:41.893127 (+ 16747us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:41.893143 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 09:33:41.893337 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:33:41.894119 (+ 782us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:33:41.894123 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:33:41.894167 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:33:41.894646 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:33:41.894655 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:33:41.896712 (+ 2057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:33:41.896718 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:33:41.899314 (+ 2596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:33:41.899331 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:33:41.900905 (+ 1574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:33:41.900911 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:33:41.901108 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:41.901138 (+ 30us) client_negotiation.cc:769] Sending connection context
0806 09:33:41.901241 (+ 103us) client_negotiation.cc:240] Negotiation successful
0806 09:33:41.901402 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16632,"mutex_wait_us":26,"spinlock_wait_cycles":1280,"thread_start_us":16157,"threads_started":1}
I20260806 09:33:41.901887 16478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:41.877039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54012 (local address 127.20.113.1:39695)
0806 09:33:41.886981 (+ 9942us) server_negotiation.cc:206] Beginning negotiation
0806 09:33:41.886989 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 09:33:41.893228 (+ 6239us) server_negotiation.cc:407] Connection header received
0806 09:33:41.893554 (+ 326us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:33:41.893562 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:33:41.893781 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:33:41.893948 (+ 167us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:33:41.894883 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:41.896548 (+ 1665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:41.899548 (+ 3000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:41.900677 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:41.900830 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:41.901128 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0806 09:33:41.901318 (+ 190us) server_negotiation.cc:299] Negotiation successful
0806 09:33:41.901517 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9752,"thread_start_us":11260,"threads_started":1}
I20260806 09:33:41.902798 16476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:41.876739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43560 (local address 127.20.113.3:40437)
0806 09:33:41.885483 (+ 8744us) server_negotiation.cc:206] Beginning negotiation
0806 09:33:41.885495 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 09:33:41.892990 (+ 7495us) server_negotiation.cc:407] Connection header received
0806 09:33:41.893322 (+ 332us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:33:41.893333 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:33:41.893581 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:33:41.893772 (+ 191us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:33:41.894998 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:41.896640 (+ 1642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:41.899536 (+ 2896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:41.900741 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:41.900911 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:41.901228 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0806 09:33:41.902239 (+ 1011us) server_negotiation.cc:299] Negotiation successful
0806 09:33:41.902433 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8527,"thread_start_us":8364,"threads_started":1}
I20260806 09:33:41.903272 16475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:41.875945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.3:40437 (local address 127.0.0.1:43560)
0806 09:33:41.892845 (+ 16900us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:41.892871 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 09:33:41.893100 (+ 229us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:33:41.894054 (+ 954us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:33:41.894061 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:33:41.894087 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:33:41.894751 (+ 664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:33:41.894764 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:33:41.896761 (+ 1997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:33:41.896770 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:33:41.899260 (+ 2490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:33:41.899282 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:33:41.901372 (+ 2090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:33:41.901377 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:33:41.901576 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:41.901607 (+ 31us) client_negotiation.cc:769] Sending connection context
0806 09:33:41.901699 (+ 92us) client_negotiation.cc:240] Negotiation successful
0806 09:33:41.901857 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16748,"mutex_wait_us":272,"spinlock_wait_cycles":1280,"thread_start_us":9642,"threads_started":1}
I20260806 09:33:41.903576 16171 tablet_service.cc:1467] Processing CreateTablet for tablet 6589a274d91649bb9102e31b391ec22e (DEFAULT_TABLE table=test-table [id=a42ed69ed4e44d4d94501b4cad43b650]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:33:41.905136 16171 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6589a274d91649bb9102e31b391ec22e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:33:41.905761 15982 tablet_service.cc:1467] Processing CreateTablet for tablet 6589a274d91649bb9102e31b391ec22e (DEFAULT_TABLE table=test-table [id=a42ed69ed4e44d4d94501b4cad43b650]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:33:41.906265 16358 tablet_service.cc:1467] Processing CreateTablet for tablet 6589a274d91649bb9102e31b391ec22e (DEFAULT_TABLE table=test-table [id=a42ed69ed4e44d4d94501b4cad43b650]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:33:41.907228 15982 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6589a274d91649bb9102e31b391ec22e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:33:41.907557 16358 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6589a274d91649bb9102e31b391ec22e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:33:41.908800 16078 heartbeater.cc:503] Master 127.20.113.62:37553 was elected leader, sending a full tablet report...
I20260806 09:33:41.920084 16479 tablet_bootstrap.cc:492] T 6589a274d91649bb9102e31b391ec22e P 72cb1184a446423db895b3bf2109231e: Bootstrap starting.
I20260806 09:33:41.921411 16480 tablet_bootstrap.cc:492] T 6589a274d91649bb9102e31b391ec22e P bdbf625fbcda46ec9a5f9a0faadaa31c: Bootstrap starting.
I20260806 09:33:41.923130 16482 tablet_bootstrap.cc:492] T 6589a274d91649bb9102e31b391ec22e P 6f7ceb74338248eaa57273407aaaab3f: Bootstrap starting.
I20260806 09:33:41.925061 16480 tablet_bootstrap.cc:654] T 6589a274d91649bb9102e31b391ec22e P bdbf625fbcda46ec9a5f9a0faadaa31c: Neither blocks nor log segments found. Creating new log.
I20260806 09:33:41.925061 16479 tablet_bootstrap.cc:654] T 6589a274d91649bb9102e31b391ec22e P 72cb1184a446423db895b3bf2109231e: Neither blocks nor log segments found. Creating new log.
I20260806 09:33:41.926339 16482 tablet_bootstrap.cc:654] T 6589a274d91649bb9102e31b391ec22e P 6f7ceb74338248eaa57273407aaaab3f: Neither blocks nor log segments found. Creating new log.
I20260806 09:33:41.926430 16480 log.cc:824] T 6589a274d91649bb9102e31b391ec22e P bdbf625fbcda46ec9a5f9a0faadaa31c: Log is configured to *not* fsync() on all Append() calls
I20260806 09:33:41.926430 16479 log.cc:824] T 6589a274d91649bb9102e31b391ec22e P 72cb1184a446423db895b3bf2109231e: Log is configured to *not* fsync() on all Append() calls
I20260806 09:33:41.927573 16482 log.cc:824] T 6589a274d91649bb9102e31b391ec22e P 6f7ceb74338248eaa57273407aaaab3f: Log is configured to *not* fsync() on all Append() calls
I20260806 09:33:41.929831 16480 tablet_bootstrap.cc:492] T 6589a274d91649bb9102e31b391ec22e P bdbf625fbcda46ec9a5f9a0faadaa31c: No bootstrap required, opened a new log
I20260806 09:33:41.929843 16479 tablet_bootstrap.cc:492] T 6589a274d91649bb9102e31b391ec22e P 72cb1184a446423db895b3bf2109231e: No bootstrap required, opened a new log
I20260806 09:33:41.930117 16482 tablet_bootstrap.cc:492] T 6589a274d91649bb9102e31b391ec22e P 6f7ceb74338248eaa57273407aaaab3f: No bootstrap required, opened a new log
I20260806 09:33:41.930127 16479 ts_tablet_manager.cc:1397] T 6589a274d91649bb9102e31b391ec22e P 72cb1184a446423db895b3bf2109231e: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260806 09:33:41.930127 16480 ts_tablet_manager.cc:1397] T 6589a274d91649bb9102e31b391ec22e P bdbf625fbcda46ec9a5f9a0faadaa31c: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260806 09:33:41.930375 16482 ts_tablet_manager.cc:1397] T 6589a274d91649bb9102e31b391ec22e P 6f7ceb74338248eaa57273407aaaab3f: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.005s
I20260806 09:33:41.938069 16482 raft_consensus.cc:348] T 6589a274d91649bb9102e31b391ec22e P 6f7ceb74338248eaa57273407aaaab3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdbf625fbcda46ec9a5f9a0faadaa31c" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 33525 } } peers { permanent_uuid: "72cb1184a446423db895b3bf2109231e" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 40437 } } peers { permanent_uuid: "6f7ceb74338248eaa57273407aaaab3f" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 39695 } }
I20260806 09:33:41.938509 16482 raft_consensus.cc:374] T 6589a274d91649bb9102e31b391ec22e P 6f7ceb74338248eaa57273407aaaab3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:33:41.938642 16482 raft_consensus.cc:729] T 6589a274d91649bb9102e31b391ec22e P 6f7ceb74338248eaa57273407aaaab3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f7ceb74338248eaa57273407aaaab3f, State: Initialized, Role: FOLLOWER
I20260806 09:33:41.939296 16482 consensus_queue.cc:260] T 6589a274d91649bb9102e31b391ec22e P 6f7ceb74338248eaa57273407aaaab3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdbf625fbcda46ec9a5f9a0faadaa31c" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 33525 } } peers { permanent_uuid: "72cb1184a446423db895b3bf2109231e" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 40437 } } peers { permanent_uuid: "6f7ceb74338248eaa57273407aaaab3f" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 39695 } }
I20260806 09:33:41.941144 16482 ts_tablet_manager.cc:1428] T 6589a274d91649bb9102e31b391ec22e P 6f7ceb74338248eaa57273407aaaab3f: Time spent starting tablet: real 0.011s user 0.011s sys 0.000s
I20260806 09:33:41.940969 16480 raft_consensus.cc:348] T 6589a274d91649bb9102e31b391ec22e P bdbf625fbcda46ec9a5f9a0faadaa31c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdbf625fbcda46ec9a5f9a0faadaa31c" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 33525 } } peers { permanent_uuid: "72cb1184a446423db895b3bf2109231e" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 40437 } } peers { permanent_uuid: "6f7ceb74338248eaa57273407aaaab3f" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 39695 } }
I20260806 09:33:41.941440 16480 raft_consensus.cc:374] T 6589a274d91649bb9102e31b391ec22e P bdbf625fbcda46ec9a5f9a0faadaa31c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:33:41.941548 16480 raft_consensus.cc:729] T 6589a274d91649bb9102e31b391ec22e P bdbf625fbcda46ec9a5f9a0faadaa31c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdbf625fbcda46ec9a5f9a0faadaa31c, State: Initialized, Role: FOLLOWER
I20260806 09:33:41.941408 16479 raft_consensus.cc:348] T 6589a274d91649bb9102e31b391ec22e P 72cb1184a446423db895b3bf2109231e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdbf625fbcda46ec9a5f9a0faadaa31c" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 33525 } } peers { permanent_uuid: "72cb1184a446423db895b3bf2109231e" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 40437 } } peers { permanent_uuid: "6f7ceb74338248eaa57273407aaaab3f" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 39695 } }
I20260806 09:33:41.941957 16479 raft_consensus.cc:374] T 6589a274d91649bb9102e31b391ec22e P 72cb1184a446423db895b3bf2109231e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:33:41.942108 16479 raft_consensus.cc:729] T 6589a274d91649bb9102e31b391ec22e P 72cb1184a446423db895b3bf2109231e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72cb1184a446423db895b3bf2109231e, State: Initialized, Role: FOLLOWER
I20260806 09:33:41.942349 16480 consensus_queue.cc:260] T 6589a274d91649bb9102e31b391ec22e P bdbf625fbcda46ec9a5f9a0faadaa31c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdbf625fbcda46ec9a5f9a0faadaa31c" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 33525 } } peers { permanent_uuid: "72cb1184a446423db895b3bf2109231e" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 40437 } } peers { permanent_uuid: "6f7ceb74338248eaa57273407aaaab3f" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 39695 } }
I20260806 09:33:41.942698 16479 consensus_queue.cc:260] T 6589a274d91649bb9102e31b391ec22e P 72cb1184a446423db895b3bf2109231e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdbf625fbcda46ec9a5f9a0faadaa31c" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 33525 } } peers { permanent_uuid: "72cb1184a446423db895b3bf2109231e" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 40437 } } peers { permanent_uuid: "6f7ceb74338248eaa57273407aaaab3f" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 39695 } }
I20260806 09:33:41.943558 16265 heartbeater.cc:503] Master 127.20.113.62:37553 was elected leader, sending a full tablet report...
I20260806 09:33:41.947494 16480 ts_tablet_manager.cc:1428] T 6589a274d91649bb9102e31b391ec22e P bdbf625fbcda46ec9a5f9a0faadaa31c: Time spent starting tablet: real 0.017s user 0.015s sys 0.000s
I20260806 09:33:41.948587 16458 heartbeater.cc:503] Master 127.20.113.62:37553 was elected leader, sending a full tablet report...
I20260806 09:33:41.949219 16479 ts_tablet_manager.cc:1428] T 6589a274d91649bb9102e31b391ec22e P 72cb1184a446423db895b3bf2109231e: Time spent starting tablet: real 0.019s user 0.016s sys 0.004s
I20260806 09:33:41.966041 16488 raft_consensus.cc:482] T 6589a274d91649bb9102e31b391ec22e P 72cb1184a446423db895b3bf2109231e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 09:33:41.966289 16488 raft_consensus.cc:504] T 6589a274d91649bb9102e31b391ec22e P 72cb1184a446423db895b3bf2109231e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdbf625fbcda46ec9a5f9a0faadaa31c" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 33525 } } peers { permanent_uuid: "72cb1184a446423db895b3bf2109231e" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 40437 } } peers { permanent_uuid: "6f7ceb74338248eaa57273407aaaab3f" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 39695 } }
I20260806 09:33:41.967947 16488 leader_election.cc:288] T 6589a274d91649bb9102e31b391ec22e P 72cb1184a446423db895b3bf2109231e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bdbf625fbcda46ec9a5f9a0faadaa31c (127.20.113.2:33525), 6f7ceb74338248eaa57273407aaaab3f (127.20.113.1:39695)
I20260806 09:33:41.974234 16474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:41.968268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.3:42117 (local address 127.20.113.2:33525)
0806 09:33:41.968506 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0806 09:33:41.968512 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:33:41.968533 (+ 21us) server_negotiation.cc:407] Connection header received
0806 09:33:41.968703 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:33:41.968711 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:33:41.968807 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:33:41.968923 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:33:41.969551 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:41.970240 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:41.972376 (+ 2136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:41.973449 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:41.973626 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:41.973673 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0806 09:33:41.973950 (+ 277us) server_negotiation.cc:299] Negotiation successful
0806 09:33:41.974035 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260806 09:33:41.974368 16462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:41.968144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.2:33525 (local address 127.20.113.3:42117)
0806 09:33:41.968378 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:41.968404 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 09:33:41.968535 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:33:41.969118 (+ 583us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:33:41.969128 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:33:41.969160 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:33:41.969389 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:33:41.969404 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:33:41.970405 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:33:41.970413 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:33:41.972213 (+ 1800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:33:41.972227 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:33:41.973622 (+ 1395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:33:41.973626 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:33:41.973782 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:41.973829 (+ 47us) client_negotiation.cc:769] Sending connection context
0806 09:33:41.973921 (+ 92us) client_negotiation.cc:240] Negotiation successful
0806 09:33:41.974054 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48,"mutex_wait_us":23}
I20260806 09:33:41.975391 16198 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6589a274d91649bb9102e31b391ec22e" candidate_uuid: "72cb1184a446423db895b3bf2109231e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdbf625fbcda46ec9a5f9a0faadaa31c" is_pre_election: true
I20260806 09:33:41.975715 16478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:41.968785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.3:59299 (local address 127.20.113.1:39695)
0806 09:33:41.969000 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0806 09:33:41.969008 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 09:33:41.969465 (+ 457us) server_negotiation.cc:407] Connection header received
0806 09:33:41.969619 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:33:41.969623 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:33:41.969688 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:33:41.969783 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:33:41.970287 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:41.971416 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:41.973615 (+ 2199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:41.974828 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:41.975001 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:41.975075 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 09:33:41.975377 (+ 302us) server_negotiation.cc:299] Negotiation successful
0806 09:33:41.975487 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 09:33:41.975759 16489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:41.968482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.1:39695 (local address 127.20.113.3:59299)
0806 09:33:41.969357 (+ 875us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:41.969373 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 09:33:41.969495 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:33:41.969822 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:33:41.969826 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:33:41.969848 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:33:41.970132 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:33:41.970139 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:33:41.971572 (+ 1433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:33:41.971576 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:33:41.973427 (+ 1851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:33:41.973442 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:33:41.974989 (+ 1547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:33:41.974994 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:33:41.975154 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:41.975174 (+ 20us) client_negotiation.cc:769] Sending connection context
0806 09:33:41.975322 (+ 148us) client_negotiation.cc:240] Negotiation successful
0806 09:33:41.975438 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":755,"mutex_wait_us":1,"thread_start_us":553,"threads_started":1}
I20260806 09:33:41.975908 16198 raft_consensus.cc:2454] T 6589a274d91649bb9102e31b391ec22e P bdbf625fbcda46ec9a5f9a0faadaa31c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 72cb1184a446423db895b3bf2109231e in term 0.
I20260806 09:33:41.976668 16291 leader_election.cc:302] T 6589a274d91649bb9102e31b391ec22e P 72cb1184a446423db895b3bf2109231e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 72cb1184a446423db895b3bf2109231e, bdbf625fbcda46ec9a5f9a0faadaa31c; no voters:
I20260806 09:33:41.977123 16488 raft_consensus.cc:2781] T 6589a274d91649bb9102e31b391ec22e P 72cb1184a446423db895b3bf2109231e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 09:33:41.977275 16488 raft_consensus.cc:482] T 6589a274d91649bb9102e31b391ec22e P 72cb1184a446423db895b3bf2109231e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 09:33:41.977416 16488 raft_consensus.cc:3037] T 6589a274d91649bb9102e31b391ec22e P 72cb1184a446423db895b3bf2109231e [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:33:41.977260 16011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6589a274d91649bb9102e31b391ec22e" candidate_uuid: "72cb1184a446423db895b3bf2109231e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f7ceb74338248eaa57273407aaaab3f" is_pre_election: true
I20260806 09:33:41.977823 16011 raft_consensus.cc:2454] T 6589a274d91649bb9102e31b391ec22e P 6f7ceb74338248eaa57273407aaaab3f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 72cb1184a446423db895b3bf2109231e in term 0.
I20260806 09:33:41.979836 16488 raft_consensus.cc:504] T 6589a274d91649bb9102e31b391ec22e P 72cb1184a446423db895b3bf2109231e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdbf625fbcda46ec9a5f9a0faadaa31c" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 33525 } } peers { permanent_uuid: "72cb1184a446423db895b3bf2109231e" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 40437 } } peers { permanent_uuid: "6f7ceb74338248eaa57273407aaaab3f" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 39695 } }
I20260806 09:33:41.980497 16488 leader_election.cc:288] T 6589a274d91649bb9102e31b391ec22e P 72cb1184a446423db895b3bf2109231e [CANDIDATE]: Term 1 election: Requested vote from peers bdbf625fbcda46ec9a5f9a0faadaa31c (127.20.113.2:33525), 6f7ceb74338248eaa57273407aaaab3f (127.20.113.1:39695)
I20260806 09:33:41.980834 16198 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6589a274d91649bb9102e31b391ec22e" candidate_uuid: "72cb1184a446423db895b3bf2109231e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdbf625fbcda46ec9a5f9a0faadaa31c"
I20260806 09:33:41.980985 16011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6589a274d91649bb9102e31b391ec22e" candidate_uuid: "72cb1184a446423db895b3bf2109231e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f7ceb74338248eaa57273407aaaab3f"
I20260806 09:33:41.981015 16198 raft_consensus.cc:3037] T 6589a274d91649bb9102e31b391ec22e P bdbf625fbcda46ec9a5f9a0faadaa31c [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:33:41.981197 16011 raft_consensus.cc:3037] T 6589a274d91649bb9102e31b391ec22e P 6f7ceb74338248eaa57273407aaaab3f [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:33:41.983333 16198 raft_consensus.cc:2454] T 6589a274d91649bb9102e31b391ec22e P bdbf625fbcda46ec9a5f9a0faadaa31c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 72cb1184a446423db895b3bf2109231e in term 1.
I20260806 09:33:41.983429 16011 raft_consensus.cc:2454] T 6589a274d91649bb9102e31b391ec22e P 6f7ceb74338248eaa57273407aaaab3f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 72cb1184a446423db895b3bf2109231e in term 1.
I20260806 09:33:41.983711 16291 leader_election.cc:302] T 6589a274d91649bb9102e31b391ec22e P 72cb1184a446423db895b3bf2109231e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 72cb1184a446423db895b3bf2109231e, bdbf625fbcda46ec9a5f9a0faadaa31c; no voters:
I20260806 09:33:41.983990 16488 raft_consensus.cc:2781] T 6589a274d91649bb9102e31b391ec22e P 72cb1184a446423db895b3bf2109231e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:33:41.984524 16488 raft_consensus.cc:686] T 6589a274d91649bb9102e31b391ec22e P 72cb1184a446423db895b3bf2109231e [term 1 LEADER]: Becoming Leader. State: Replica: 72cb1184a446423db895b3bf2109231e, State: Running, Role: LEADER
I20260806 09:33:41.985098 16488 consensus_queue.cc:237] T 6589a274d91649bb9102e31b391ec22e P 72cb1184a446423db895b3bf2109231e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdbf625fbcda46ec9a5f9a0faadaa31c" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 33525 } } peers { permanent_uuid: "72cb1184a446423db895b3bf2109231e" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 40437 } } peers { permanent_uuid: "6f7ceb74338248eaa57273407aaaab3f" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 39695 } }
I20260806 09:33:41.989471 15801 catalog_manager.cc:5495] T 6589a274d91649bb9102e31b391ec22e P 72cb1184a446423db895b3bf2109231e reported cstate change: term changed from 0 to 1, leader changed from <none> to 72cb1184a446423db895b3bf2109231e (127.20.113.3). New cstate: current_term: 1 leader_uuid: "72cb1184a446423db895b3bf2109231e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdbf625fbcda46ec9a5f9a0faadaa31c" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 33525 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "72cb1184a446423db895b3bf2109231e" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 40437 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6f7ceb74338248eaa57273407aaaab3f" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 39695 } health_report { overall_health: UNKNOWN } } }
I20260806 09:33:42.020921 16476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:42.015912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43566 (local address 127.20.113.3:40437)
0806 09:33:42.016088 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0806 09:33:42.016092 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 09:33:42.016110 (+ 18us) server_negotiation.cc:407] Connection header received
0806 09:33:42.016271 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:33:42.016274 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:33:42.016408 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:33:42.016532 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 09:33:42.016881 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:42.017617 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:42.018951 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:42.019360 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:42.019458 (+ 98us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:42.019899 (+ 441us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 09:33:42.020432 (+ 533us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 09:33:42.020567 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0806 09:33:42.020704 (+ 137us) server_negotiation.cc:299] Negotiation successful
0806 09:33:42.020738 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 09:33:42.028935 16011 raft_consensus.cc:1261] T 6589a274d91649bb9102e31b391ec22e P 6f7ceb74338248eaa57273407aaaab3f [term 1 FOLLOWER]: Refusing update from remote peer 72cb1184a446423db895b3bf2109231e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 09:33:42.028935 16198 raft_consensus.cc:1261] T 6589a274d91649bb9102e31b391ec22e P bdbf625fbcda46ec9a5f9a0faadaa31c [term 1 FOLLOWER]: Refusing update from remote peer 72cb1184a446423db895b3bf2109231e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 09:33:42.029908 16490 consensus_queue.cc:1035] T 6589a274d91649bb9102e31b391ec22e P 72cb1184a446423db895b3bf2109231e [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f7ceb74338248eaa57273407aaaab3f" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 39695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 09:33:42.030303 16488 consensus_queue.cc:1035] T 6589a274d91649bb9102e31b391ec22e P 72cb1184a446423db895b3bf2109231e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bdbf625fbcda46ec9a5f9a0faadaa31c" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 33525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 09:33:42.042309 16497 mvcc.cc:205] Tried to move back new op lower bound from 7315492135020093440 to 7315492134855266304. Current Snapshot: MvccSnapshot[applied={T|T < 7315492135020093440}]
I20260806 09:33:42.065033 15799 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:36008:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 09:33:42.065447 15799 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:36008:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 09:33:42.072286 15799 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 884cc132a6374291a65ef69881c3cec4: Sending DeleteTablet for 3 replicas of tablet 6589a274d91649bb9102e31b391ec22e
I20260806 09:33:42.072327 15982 tablet_service.cc:1514] Processing DeleteTablet for tablet 6589a274d91649bb9102e31b391ec22e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:33:42 UTC) from {username='slave'} at 127.0.0.1:54012
I20260806 09:33:42.073488 16171 tablet_service.cc:1514] Processing DeleteTablet for tablet 6589a274d91649bb9102e31b391ec22e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:33:42 UTC) from {username='slave'} at 127.0.0.1:44740
I20260806 09:33:42.073658 15981 tablet_service.cc:1514] Processing DeleteTablet for tablet 6589a274d91649bb9102e31b391ec22e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:33:42 UTC) from {username='slave'} at 127.0.0.1:54012
I20260806 09:33:42.074148 16358 tablet_service.cc:1514] Processing DeleteTablet for tablet 6589a274d91649bb9102e31b391ec22e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:33:42 UTC) from {username='slave'} at 127.0.0.1:43560
I20260806 09:33:42.075814 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 15895
I20260806 09:33:42.084899 16516 tablet_replica.cc:330] T 6589a274d91649bb9102e31b391ec22e P 72cb1184a446423db895b3bf2109231e: stopping tablet replica
I20260806 09:33:42.085127 16515 tablet_replica.cc:330] T 6589a274d91649bb9102e31b391ec22e P bdbf625fbcda46ec9a5f9a0faadaa31c: stopping tablet replica
I20260806 09:33:42.085680 16515 raft_consensus.cc:2229] T 6589a274d91649bb9102e31b391ec22e P bdbf625fbcda46ec9a5f9a0faadaa31c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 09:33:42.086233 16516 raft_consensus.cc:2229] T 6589a274d91649bb9102e31b391ec22e P 72cb1184a446423db895b3bf2109231e [term 1 LEADER]: Raft consensus shutting down.
I20260806 09:33:42.086293 16515 raft_consensus.cc:2258] T 6589a274d91649bb9102e31b391ec22e P bdbf625fbcda46ec9a5f9a0faadaa31c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 09:33:42.087217 16516 raft_consensus.cc:2258] T 6589a274d91649bb9102e31b391ec22e P 72cb1184a446423db895b3bf2109231e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 09:33:42.089059 16515 ts_tablet_manager.cc:1905] T 6589a274d91649bb9102e31b391ec22e P bdbf625fbcda46ec9a5f9a0faadaa31c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 09:33:42.089327 16516 ts_tablet_manager.cc:1905] T 6589a274d91649bb9102e31b391ec22e P 72cb1184a446423db895b3bf2109231e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 09:33:42.090045 16514 tablet_replica.cc:330] T 6589a274d91649bb9102e31b391ec22e P 6f7ceb74338248eaa57273407aaaab3f: stopping tablet replica
I20260806 09:33:42.090510 16514 raft_consensus.cc:2229] T 6589a274d91649bb9102e31b391ec22e P 6f7ceb74338248eaa57273407aaaab3f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 09:33:42.090997 16514 raft_consensus.cc:2258] T 6589a274d91649bb9102e31b391ec22e P 6f7ceb74338248eaa57273407aaaab3f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 09:33:42.092944 16514 ts_tablet_manager.cc:1905] T 6589a274d91649bb9102e31b391ec22e P 6f7ceb74338248eaa57273407aaaab3f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 09:33:42.095573 16478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:42.079566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54022 (local address 127.20.113.1:39695)
0806 09:33:42.079801 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0806 09:33:42.079810 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 09:33:42.087391 (+ 7581us) server_negotiation.cc:407] Connection header received
0806 09:33:42.087715 (+ 324us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:33:42.087721 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:33:42.087840 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:33:42.087925 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:33:42.088666 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:42.089788 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:42.091616 (+ 1828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:42.092261 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:42.092379 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:42.093974 (+ 1595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:33:42.093984 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:33:42.094000 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:33:42.094693 (+ 693us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:33:42.094751 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:33:42.094776 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:33:42.094883 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 09:33:42.095145 (+ 262us) server_negotiation.cc:299] Negotiation successful
0806 09:33:42.095257 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260806 09:33:42.096491 16515 ts_tablet_manager.cc:1918] T 6589a274d91649bb9102e31b391ec22e P bdbf625fbcda46ec9a5f9a0faadaa31c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 09:33:42.096729 16515 log.cc:1196] T 6589a274d91649bb9102e31b391ec22e P bdbf625fbcda46ec9a5f9a0faadaa31c: Deleting WAL directory at /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786008495763054-20932-0/minicluster-data/ts-1/wal/wals/6589a274d91649bb9102e31b391ec22e
I20260806 09:33:42.096751 16071 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:33:42.097427 16515 ts_tablet_manager.cc:1939] T 6589a274d91649bb9102e31b391ec22e P bdbf625fbcda46ec9a5f9a0faadaa31c: Deleting consensus metadata
I20260806 09:33:42.098203 16516 ts_tablet_manager.cc:1918] T 6589a274d91649bb9102e31b391ec22e P 72cb1184a446423db895b3bf2109231e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 09:33:42.098438 16516 log.cc:1196] T 6589a274d91649bb9102e31b391ec22e P 72cb1184a446423db895b3bf2109231e: Deleting WAL directory at /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786008495763054-20932-0/minicluster-data/ts-2/wal/wals/6589a274d91649bb9102e31b391ec22e
I20260806 09:33:42.099161 16516 ts_tablet_manager.cc:1939] T 6589a274d91649bb9102e31b391ec22e P 72cb1184a446423db895b3bf2109231e: Deleting consensus metadata
I20260806 09:33:42.099623 15781 catalog_manager.cc:4841] TS bdbf625fbcda46ec9a5f9a0faadaa31c (127.20.113.2:33525): tablet 6589a274d91649bb9102e31b391ec22e (table test-table [id=a42ed69ed4e44d4d94501b4cad43b650]) successfully deleted
I20260806 09:33:42.101168 15780 catalog_manager.cc:4841] TS 72cb1184a446423db895b3bf2109231e (127.20.113.3:40437): tablet 6589a274d91649bb9102e31b391ec22e (table test-table [id=a42ed69ed4e44d4d94501b4cad43b650]) successfully deleted
I20260806 09:33:42.191354 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 15895
I20260806 09:33:42.223519 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 16084
W20260806 09:33:42.223531 15785 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.20.113.1:39695 (error 108)
I20260806 09:33:42.225967 16475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:42.225424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.1:39695 (local address 127.0.0.1:48426)
0806 09:33:42.225643 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:42.225747 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.1:39695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
W20260806 09:33:42.226267 15785 catalog_manager.cc:4584] TS 6f7ceb74338248eaa57273407aaaab3f (127.20.113.1:39695): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6589a274d91649bb9102e31b391ec22e: Network error: Client connection negotiation failed: client connection to 127.20.113.1:39695: connect: Connection refused (error 111)
I20260806 09:33:42.232049 16474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:42.224836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38020 (local address 127.20.113.2:33525)
0806 09:33:42.225050 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0806 09:33:42.225058 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 09:33:42.225078 (+ 20us) server_negotiation.cc:407] Connection header received
0806 09:33:42.225146 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:33:42.225154 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:33:42.225269 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:33:42.225395 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:33:42.225912 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:42.226951 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:42.228923 (+ 1972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:42.229309 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:42.229407 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:42.230613 (+ 1206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:33:42.230622 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:33:42.230636 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:33:42.231248 (+ 612us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:33:42.231297 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:33:42.231317 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:33:42.231414 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 09:33:42.231570 (+ 156us) server_negotiation.cc:299] Negotiation successful
0806 09:33:42.231657 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260806 09:33:42.232841 16260 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:33:42.280236 16475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:42.279673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.1:39695 (local address 127.0.0.1:48432)
0806 09:33:42.279914 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:42.280025 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.1:39695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260806 09:33:42.281531 16475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:42.281089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.1:39695 (local address 127.0.0.1:48444)
0806 09:33:42.281267 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:42.281375 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.1:39695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260806 09:33:42.334532 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 16084
I20260806 09:33:42.342523 16475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:42.341931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.1:39695 (local address 127.0.0.1:48446)
0806 09:33:42.342231 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:42.342341 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.1:39695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164}
I20260806 09:33:42.343588 16475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:42.343250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.1:39695 (local address 127.0.0.1:48462)
0806 09:33:42.343411 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:42.343487 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.1:39695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260806 09:33:42.364969 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 16268
I20260806 09:33:42.377508 16476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:42.365785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54784 (local address 127.20.113.3:40437)
0806 09:33:42.365959 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0806 09:33:42.365963 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 09:33:42.365987 (+ 24us) server_negotiation.cc:407] Connection header received
0806 09:33:42.366214 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:33:42.366218 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:33:42.366347 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:33:42.366435 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:33:42.366946 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:42.367737 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:42.369860 (+ 2123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:42.370369 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:42.370470 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:42.375863 (+ 5393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:33:42.375872 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:33:42.375886 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:33:42.376656 (+ 770us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:33:42.376705 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:33:42.376727 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:33:42.376815 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 09:33:42.377022 (+ 207us) server_negotiation.cc:299] Negotiation successful
0806 09:33:42.377105 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 09:33:42.378240 16448 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:33:42.424489 16475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:42.423894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.1:39695 (local address 127.0.0.1:48478)
0806 09:33:42.424147 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:42.424274 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.1:39695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260806 09:33:42.425650 16475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:42.425187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.1:39695 (local address 127.0.0.1:48494)
0806 09:33:42.425428 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:42.425516 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.1:39695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124}
I20260806 09:33:42.505218 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 16268
I20260806 09:33:42.541030 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 15759
I20260806 09:33:42.542449 15857 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:33:42.668238 16475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:42.667333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.1:39695 (local address 127.0.0.1:48500)
0806 09:33:42.667568 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:42.667737 (+ 169us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.1:39695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
I20260806 09:33:42.668632 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 15759
2026-08-06T09:33:42Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (2923 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786008495763054-20932-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 09:33:42 dist-test-slave-45dx krb5kdc[16529](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 09:33:42 dist-test-slave-45dx krb5kdc[16529](info): set up 2 sockets
Aug 06 09:33:42 dist-test-slave-45dx krb5kdc[16529](info): commencing operation
krb5kdc: starting...
W20260806 09:33:43.975750 20932 mini_kdc.cc:121] Time spent starting KDC: real 1.256s user 0.003s sys 0.051s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786008495763054-20932-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786008495763054-20932-0/krb5kdc/test-user.keytab.
Aug 06 09:33:44 dist-test-slave-45dx krb5kdc[16529](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008824, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T09:33:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T09:33:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.113.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.113.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.113.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab.
Entry for principal kudu/127.20.113.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab.
I20260806 09:33:44.323958 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786008495763054-20932-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786008495763054-20932-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786008495763054-20932-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.113.62:40985
--webserver_interface=127.20.113.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab
--principal=kudu/127.20.113.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:39385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.113.62:40985
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:33:44.670017 16547 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:33:44.670277 16547 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:33:44.670455 16547 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:33:44.677042 16547 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 09:33:44.677131 16547 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:33:44.677212 16547 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:33:44.677332 16547 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 09:33:44.677420 16547 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 09:33:44.686589 16547 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:39385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786008495763054-20932-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786008495763054-20932-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.113.62:40985
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab
--principal=kudu/127.20.113.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.113.62:40985
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.113.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786008495763054-20932-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:33:44.687882 16547 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:33:44.689261 16547 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:33:44.701220 16561 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:33:44.701177 16559 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:33:44.701918 16558 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:33:44.702236 16547 server_base.cc:1034] running on GCE node
I20260806 09:33:44.703058 16547 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:33:44.704444 16547 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:33:44.705690 16547 hybrid_clock.cc:648] HybridClock initialized: now 1786008824705643 us; error 28 us; skew 500 ppm
I20260806 09:33:44.706103 16547 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:33:44 dist-test-slave-45dx krb5kdc[16529](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008824, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:33:44.714737 16547 init.cc:377] Logged in from keytab as kudu/127.20.113.62@KRBTEST.COM (short username kudu)
I20260806 09:33:44.716780 16547 webserver.cc:460] Webserver started at http://127.20.113.62:35543/ using document root <none> and password file <none>
I20260806 09:33:44.717487 16547 fs_manager.cc:359] Metadata directory not provided
I20260806 09:33:44.717648 16547 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:33:44.718000 16547 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:33:44.721164 16547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786008495763054-20932-0/minicluster-data/master-0/data/instance:
uuid: "932b787c8f99437bab3c970e2b7689db"
format_stamp: "Formatted at 2026-08-06 09:33:44 on dist-test-slave-45dx"
server_key: "c4d5b99a540d6b508fdef479765cc991"
server_key_iv: "601faf76500045e26c3197c1e72de307"
server_key_version: "encryptionkey@0"
I20260806 09:33:44.721999 16547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786008495763054-20932-0/minicluster-data/master-0/wal/instance:
uuid: "932b787c8f99437bab3c970e2b7689db"
format_stamp: "Formatted at 2026-08-06 09:33:44 on dist-test-slave-45dx"
server_key: "c4d5b99a540d6b508fdef479765cc991"
server_key_iv: "601faf76500045e26c3197c1e72de307"
server_key_version: "encryptionkey@0"
I20260806 09:33:44.727139 16547 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260806 09:33:44.730715 16572 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:33:44.732015 16547 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260806 09:33:44.732198 16547 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786008495763054-20932-0/minicluster-data/master-0/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786008495763054-20932-0/minicluster-data/master-0/wal
uuid: "932b787c8f99437bab3c970e2b7689db"
format_stamp: "Formatted at 2026-08-06 09:33:44 on dist-test-slave-45dx"
server_key: "c4d5b99a540d6b508fdef479765cc991"
server_key_iv: "601faf76500045e26c3197c1e72de307"
server_key_version: "encryptionkey@0"
I20260806 09:33:44.732460 16547 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786008495763054-20932-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786008495763054-20932-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786008495763054-20932-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:33:44.749018 16547 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:33:44.753123 16547 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:33:44.753412 16547 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:33:44.771545 16547 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.62:40985
I20260806 09:33:44.771569 16642 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.62:40985 every 8 connection(s)
I20260806 09:33:44.772969 16547 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
I20260806 09:33:44.776578 16643 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:33:44.780032 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 16547
I20260806 09:33:44.780223 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786008495763054-20932-0/minicluster-data/master-0/wal/instance
I20260806 09:33:44.780821 20932 external_mini_cluster.cc:1428] Setting key eeff93b07e27417aa5f4de535c76e3bb
I20260806 09:33:44.789281 16643 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 932b787c8f99437bab3c970e2b7689db: Bootstrap starting.
I20260806 09:33:44.793002 16643 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 932b787c8f99437bab3c970e2b7689db: Neither blocks nor log segments found. Creating new log.
I20260806 09:33:44.794322 16643 log.cc:824] T 00000000000000000000000000000000 P 932b787c8f99437bab3c970e2b7689db: Log is configured to *not* fsync() on all Append() calls
Aug 06 09:33:44 dist-test-slave-45dx krb5kdc[16529](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008824, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.113.62@KRBTEST.COM
I20260806 09:33:44.797127 16643 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 932b787c8f99437bab3c970e2b7689db: No bootstrap required, opened a new log
I20260806 09:33:44.803799 16648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:44.782668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60170 (local address 127.20.113.62:40985)
0806 09:33:44.783418 (+ 750us) server_negotiation.cc:206] Beginning negotiation
0806 09:33:44.783428 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 09:33:44.783468 (+ 40us) server_negotiation.cc:407] Connection header received
0806 09:33:44.785179 (+ 1711us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:33:44.785200 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:33:44.785582 (+ 382us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:33:44.785887 (+ 305us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:33:44.787679 (+ 1792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:44.789010 (+ 1331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:44.790757 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:44.791412 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:44.791532 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:44.796449 (+ 4917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:33:44.796456 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:33:44.796469 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:33:44.796522 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:33:44.799737 (+ 3215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:33:44.800555 (+ 818us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:33:44.800567 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:33:44.800572 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:33:44.800731 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:33:44.801140 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:33:44.801146 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:33:44.801156 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:33:44.801657 (+ 501us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:33:44.802004 (+ 347us) server_negotiation.cc:1035] Waiting for connection context
0806 09:33:44.802440 (+ 436us) server_negotiation.cc:299] Negotiation successful
0806 09:33:44.802844 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":560,"thread_start_us":358,"threads_started":1}
I20260806 09:33:44.804744 16643 raft_consensus.cc:348] T 00000000000000000000000000000000 P 932b787c8f99437bab3c970e2b7689db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "932b787c8f99437bab3c970e2b7689db" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 40985 } }
I20260806 09:33:44.805104 16643 raft_consensus.cc:374] T 00000000000000000000000000000000 P 932b787c8f99437bab3c970e2b7689db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:33:44.805244 16643 raft_consensus.cc:729] T 00000000000000000000000000000000 P 932b787c8f99437bab3c970e2b7689db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 932b787c8f99437bab3c970e2b7689db, State: Initialized, Role: FOLLOWER
I20260806 09:33:44.805763 16643 consensus_queue.cc:260] T 00000000000000000000000000000000 P 932b787c8f99437bab3c970e2b7689db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "932b787c8f99437bab3c970e2b7689db" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 40985 } }
I20260806 09:33:44.806048 16643 raft_consensus.cc:388] T 00000000000000000000000000000000 P 932b787c8f99437bab3c970e2b7689db [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 09:33:44.806200 16643 raft_consensus.cc:482] T 00000000000000000000000000000000 P 932b787c8f99437bab3c970e2b7689db [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 09:33:44.806401 16643 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 932b787c8f99437bab3c970e2b7689db [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:33:44.808694 16643 raft_consensus.cc:504] T 00000000000000000000000000000000 P 932b787c8f99437bab3c970e2b7689db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "932b787c8f99437bab3c970e2b7689db" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 40985 } }
I20260806 09:33:44.809199 16643 leader_election.cc:302] T 00000000000000000000000000000000 P 932b787c8f99437bab3c970e2b7689db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 932b787c8f99437bab3c970e2b7689db; no voters:
I20260806 09:33:44.809762 16643 leader_election.cc:288] T 00000000000000000000000000000000 P 932b787c8f99437bab3c970e2b7689db [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 09:33:44.809893 16652 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 932b787c8f99437bab3c970e2b7689db [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:33:44.811235 16652 raft_consensus.cc:686] T 00000000000000000000000000000000 P 932b787c8f99437bab3c970e2b7689db [term 1 LEADER]: Becoming Leader. State: Replica: 932b787c8f99437bab3c970e2b7689db, State: Running, Role: LEADER
I20260806 09:33:44.811756 16652 consensus_queue.cc:237] T 00000000000000000000000000000000 P 932b787c8f99437bab3c970e2b7689db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "932b787c8f99437bab3c970e2b7689db" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 40985 } }
I20260806 09:33:44.811875 16643 sys_catalog.cc:564] T 00000000000000000000000000000000 P 932b787c8f99437bab3c970e2b7689db [sys.catalog]: configured and running, proceeding with master startup.
I20260806 09:33:44.814633 16655 sys_catalog.cc:455] T 00000000000000000000000000000000 P 932b787c8f99437bab3c970e2b7689db [sys.catalog]: SysCatalogTable state changed. Reason: New leader 932b787c8f99437bab3c970e2b7689db. Latest consensus state: current_term: 1 leader_uuid: "932b787c8f99437bab3c970e2b7689db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "932b787c8f99437bab3c970e2b7689db" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 40985 } } }
I20260806 09:33:44.815052 16655 sys_catalog.cc:458] T 00000000000000000000000000000000 P 932b787c8f99437bab3c970e2b7689db [sys.catalog]: This master's current role is: LEADER
I20260806 09:33:44.815534 16654 sys_catalog.cc:455] T 00000000000000000000000000000000 P 932b787c8f99437bab3c970e2b7689db [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "932b787c8f99437bab3c970e2b7689db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "932b787c8f99437bab3c970e2b7689db" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 40985 } } }
I20260806 09:33:44.815825 16654 sys_catalog.cc:458] T 00000000000000000000000000000000 P 932b787c8f99437bab3c970e2b7689db [sys.catalog]: This master's current role is: LEADER
I20260806 09:33:44.818214 16664 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 09:33:44.822759 16664 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 09:33:44.831502 16664 catalog_manager.cc:1347] Generated new cluster ID: 9c980ba3df8a4b73afbf03ad28fdefeb
I20260806 09:33:44.831616 16664 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 09:33:44.841768 16664 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 09:33:44.843616 16664 catalog_manager.cc:1504] Loading token signing keys...
I20260806 09:33:44.858932 16664 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 932b787c8f99437bab3c970e2b7689db: Generated new TSK 0
I20260806 09:33:44.859757 16664 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.113.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.113.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.113.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.1.keytab.
Entry for principal kudu/127.20.113.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.1.keytab.
I20260806 09:33:44.959133 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786008495763054-20932-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786008495763054-20932-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786008495763054-20932-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786008495763054-20932-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.113.1:0
--local_ip_for_outbound_sockets=127.20.113.1
--webserver_interface=127.20.113.1
--webserver_port=0
--tserver_master_addrs=127.20.113.62:40985
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.1.keytab
--principal=kudu/127.20.113.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:39385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:33:45.312368 16686 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:33:45.312703 16686 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:33:45.312909 16686 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:33:45.320281 16686 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:33:45.320462 16686 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:33:45.320654 16686 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.113.1
I20260806 09:33:45.330606 16686 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:39385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786008495763054-20932-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786008495763054-20932-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.1.keytab
--principal=kudu/127.20.113.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.113.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786008495763054-20932-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.113.1
--webserver_port=0
--tserver_master_addrs=127.20.113.62:40985
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.113.1
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786008495763054-20932-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:33:45.331913 16686 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:33:45.333349 16686 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:33:45.346951 16698 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:33:45.347036 16694 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:33:45.347036 16692 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:33:45.348134 16686 server_base.cc:1034] running on GCE node
I20260806 09:33:45.348757 16686 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:33:45.349737 16686 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:33:45.350956 16686 hybrid_clock.cc:648] HybridClock initialized: now 1786008825350916 us; error 40 us; skew 500 ppm
I20260806 09:33:45.351236 16686 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:33:45 dist-test-slave-45dx krb5kdc[16529](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008825, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:33:45.354401 16686 init.cc:377] Logged in from keytab as kudu/127.20.113.1@KRBTEST.COM (short username kudu)
I20260806 09:33:45.355825 16686 webserver.cc:460] Webserver started at http://127.20.113.1:35947/ using document root <none> and password file <none>
I20260806 09:33:45.356591 16686 fs_manager.cc:359] Metadata directory not provided
I20260806 09:33:45.356752 16686 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:33:45.357086 16686 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:33:45.360039 16686 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786008495763054-20932-0/minicluster-data/ts-0/data/instance:
uuid: "8c95494edfe44953baf6c42b6c1a1e7e"
format_stamp: "Formatted at 2026-08-06 09:33:45 on dist-test-slave-45dx"
server_key: "fb58d2fcd3766ffc40214e9a13b97a5d"
server_key_iv: "c6c1347038a8715b73644d16ee82eb90"
server_key_version: "encryptionkey@0"
I20260806 09:33:45.360888 16686 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786008495763054-20932-0/minicluster-data/ts-0/wal/instance:
uuid: "8c95494edfe44953baf6c42b6c1a1e7e"
format_stamp: "Formatted at 2026-08-06 09:33:45 on dist-test-slave-45dx"
server_key: "fb58d2fcd3766ffc40214e9a13b97a5d"
server_key_iv: "c6c1347038a8715b73644d16ee82eb90"
server_key_version: "encryptionkey@0"
I20260806 09:33:45.365996 16686 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 09:33:45.369431 16707 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:33:45.370694 16686 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 09:33:45.370878 16686 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786008495763054-20932-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786008495763054-20932-0/minicluster-data/ts-0/wal
uuid: "8c95494edfe44953baf6c42b6c1a1e7e"
format_stamp: "Formatted at 2026-08-06 09:33:45 on dist-test-slave-45dx"
server_key: "fb58d2fcd3766ffc40214e9a13b97a5d"
server_key_iv: "c6c1347038a8715b73644d16ee82eb90"
server_key_version: "encryptionkey@0"
I20260806 09:33:45.371086 16686 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786008495763054-20932-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786008495763054-20932-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786008495763054-20932-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:33:45.388177 16686 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:33:45.392198 16686 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:33:45.392589 16686 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:33:45.393590 16686 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:33:45.394938 16686 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:33:45.395040 16686 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:33:45.395196 16686 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:33:45.395269 16686 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:33:45.429867 16686 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.1:35793
I20260806 09:33:45.429912 16869 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.1:35793 every 8 connection(s)
I20260806 09:33:45.431209 16686 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786008495763054-20932-0/minicluster-data/ts-0/data/info.pb
I20260806 09:33:45.440200 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 16686
I20260806 09:33:45.440433 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786008495763054-20932-0/minicluster-data/ts-0/wal/instance
I20260806 09:33:45.440896 20932 external_mini_cluster.cc:1428] Setting key d172f8d6f95c45d66a0b64b039935077
Aug 06 09:33:45 dist-test-slave-45dx krb5kdc[16529](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008825, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.1@KRBTEST.COM for kudu/127.20.113.62@KRBTEST.COM
I20260806 09:33:45.454226 16876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:45.433807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.1:41431 (local address 127.20.113.62:40985)
0806 09:33:45.434296 (+ 489us) server_negotiation.cc:206] Beginning negotiation
0806 09:33:45.434302 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:33:45.435374 (+ 1072us) server_negotiation.cc:407] Connection header received
0806 09:33:45.437358 (+ 1984us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:33:45.437363 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:33:45.437447 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:33:45.437553 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:33:45.439018 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:45.439744 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:45.441494 (+ 1750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:45.442060 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:45.442161 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:45.445964 (+ 3803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:33:45.445973 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:33:45.445981 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:33:45.446027 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:33:45.449885 (+ 3858us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:33:45.450913 (+ 1028us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:33:45.450920 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:33:45.450925 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:33:45.451101 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:33:45.451730 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:33:45.451737 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:33:45.451741 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:33:45.452180 (+ 439us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:33:45.452368 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0806 09:33:45.453538 (+ 1170us) server_negotiation.cc:299] Negotiation successful
0806 09:33:45.453795 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":361,"thread_start_us":222,"threads_started":1}
I20260806 09:33:45.455803 16880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:45.434415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:40985 (local address 127.20.113.1:41431)
0806 09:33:45.435144 (+ 729us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:45.435241 (+ 97us) client_negotiation.cc:174] Beginning negotiation
0806 09:33:45.437109 (+ 1868us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:33:45.437771 (+ 662us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:33:45.437778 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:33:45.438295 (+ 517us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:33:45.438870 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:33:45.438887 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:33:45.439885 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:33:45.439889 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:33:45.441315 (+ 1426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:33:45.441339 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:33:45.442296 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:33:45.442304 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:33:45.442591 (+ 287us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:45.443434 (+ 843us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:33:45.443460 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:33:45.445734 (+ 2274us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:33:45.450054 (+ 4320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:33:45.450065 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:33:45.450084 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:33:45.450722 (+ 638us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:33:45.451249 (+ 527us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:33:45.451257 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:33:45.451263 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:33:45.451550 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:33:45.452581 (+ 1031us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:33:45.452593 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:33:45.453052 (+ 459us) client_negotiation.cc:769] Sending connection context
0806 09:33:45.453367 (+ 315us) client_negotiation.cc:240] Negotiation successful
0806 09:33:45.453972 (+ 605us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":288,"threads_started":1}
I20260806 09:33:45.457850 16871 heartbeater.cc:347] Connected to a master server at 127.20.113.62:40985
I20260806 09:33:45.458212 16871 heartbeater.cc:464] Registering TS with master...
I20260806 09:33:45.459089 16871 heartbeater.cc:511] Master 127.20.113.62:40985 requested a full tablet report, sending...
I20260806 09:33:45.461266 16593 ts_manager.cc:194] Registered new tserver with Master: 8c95494edfe44953baf6c42b6c1a1e7e (127.20.113.1:35793)
I20260806 09:33:45.463034 16593 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.113.1@KRBTEST.COM'} at 127.20.113.1:41431
WARNING: no policy specified for kudu/127.20.113.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.113.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.113.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.2.keytab.
Entry for principal kudu/127.20.113.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.2.keytab.
I20260806 09:33:45.521510 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786008495763054-20932-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786008495763054-20932-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786008495763054-20932-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786008495763054-20932-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.113.2:0
--local_ip_for_outbound_sockets=127.20.113.2
--webserver_interface=127.20.113.2
--webserver_port=0
--tserver_master_addrs=127.20.113.62:40985
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.2.keytab
--principal=kudu/127.20.113.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:39385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:33:45.869741 16885 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:33:45.870059 16885 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:33:45.870153 16885 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:33:45.877373 16885 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:33:45.877491 16885 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:33:45.877611 16885 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.113.2
I20260806 09:33:45.887352 16885 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:39385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786008495763054-20932-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786008495763054-20932-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.2.keytab
--principal=kudu/127.20.113.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.113.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786008495763054-20932-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.113.2
--webserver_port=0
--tserver_master_addrs=127.20.113.62:40985
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.113.2
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786008495763054-20932-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:33:45.888708 16885 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:33:45.890057 16885 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:33:45.903409 16892 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:33:45.903468 16893 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:33:45.903491 16895 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:33:45.904755 16885 server_base.cc:1034] running on GCE node
I20260806 09:33:45.905354 16885 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:33:45.906240 16885 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:33:45.907454 16885 hybrid_clock.cc:648] HybridClock initialized: now 1786008825907412 us; error 37 us; skew 500 ppm
I20260806 09:33:45.907788 16885 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:33:45 dist-test-slave-45dx krb5kdc[16529](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008825, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:33:45.911067 16885 init.cc:377] Logged in from keytab as kudu/127.20.113.2@KRBTEST.COM (short username kudu)
I20260806 09:33:45.912701 16885 webserver.cc:460] Webserver started at http://127.20.113.2:45533/ using document root <none> and password file <none>
I20260806 09:33:45.913478 16885 fs_manager.cc:359] Metadata directory not provided
I20260806 09:33:45.913582 16885 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:33:45.913856 16885 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:33:45.916882 16885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786008495763054-20932-0/minicluster-data/ts-1/data/instance:
uuid: "6c746c0cca474647a82aec2949649355"
format_stamp: "Formatted at 2026-08-06 09:33:45 on dist-test-slave-45dx"
server_key: "0d19499dc90d3d40194b483dc458e6a4"
server_key_iv: "f4ba42fe1b38d0e9959e36f87a923347"
server_key_version: "encryptionkey@0"
I20260806 09:33:45.917598 16885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786008495763054-20932-0/minicluster-data/ts-1/wal/instance:
uuid: "6c746c0cca474647a82aec2949649355"
format_stamp: "Formatted at 2026-08-06 09:33:45 on dist-test-slave-45dx"
server_key: "0d19499dc90d3d40194b483dc458e6a4"
server_key_iv: "f4ba42fe1b38d0e9959e36f87a923347"
server_key_version: "encryptionkey@0"
I20260806 09:33:45.922761 16885 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 09:33:45.926164 16905 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:33:45.927466 16885 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 09:33:45.927654 16885 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786008495763054-20932-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786008495763054-20932-0/minicluster-data/ts-1/wal
uuid: "6c746c0cca474647a82aec2949649355"
format_stamp: "Formatted at 2026-08-06 09:33:45 on dist-test-slave-45dx"
server_key: "0d19499dc90d3d40194b483dc458e6a4"
server_key_iv: "f4ba42fe1b38d0e9959e36f87a923347"
server_key_version: "encryptionkey@0"
I20260806 09:33:45.927874 16885 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786008495763054-20932-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786008495763054-20932-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786008495763054-20932-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:33:45.939745 16885 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:33:45.944684 16885 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:33:45.945113 16885 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:33:45.946193 16885 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:33:45.947540 16885 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:33:45.947616 16885 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:33:45.947705 16885 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:33:45.947749 16885 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:33:45.982703 16885 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.2:41313
I20260806 09:33:45.982748 17069 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.2:41313 every 8 connection(s)
I20260806 09:33:45.984035 16885 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786008495763054-20932-0/minicluster-data/ts-1/data/info.pb
I20260806 09:33:45.986469 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 16885
I20260806 09:33:45.986703 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786008495763054-20932-0/minicluster-data/ts-1/wal/instance
I20260806 09:33:45.987311 20932 external_mini_cluster.cc:1428] Setting key 273363b7e327176a33616217ee72cc8e
Aug 06 09:33:45 dist-test-slave-45dx krb5kdc[16529](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008825, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.2@KRBTEST.COM for kudu/127.20.113.62@KRBTEST.COM
I20260806 09:33:46.007859 17074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:45.987231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.2:48133 (local address 127.20.113.62:40985)
0806 09:33:45.988827 (+ 1596us) server_negotiation.cc:206] Beginning negotiation
0806 09:33:45.988836 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 09:33:45.989610 (+ 774us) server_negotiation.cc:407] Connection header received
0806 09:33:45.991577 (+ 1967us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:33:45.991584 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:33:45.991675 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:33:45.991816 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:33:45.993558 (+ 1742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:45.994689 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:45.996298 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:45.997062 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:45.997201 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:46.000945 (+ 3744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:33:46.000953 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:33:46.000960 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:33:46.001003 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:33:46.004697 (+ 3694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:33:46.005440 (+ 743us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:33:46.005447 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:33:46.005452 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:33:46.005625 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:33:46.006054 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:33:46.006061 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:33:46.006066 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:33:46.006431 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:33:46.006579 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0806 09:33:46.007319 (+ 740us) server_negotiation.cc:299] Negotiation successful
0806 09:33:46.007486 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1403,"thread_start_us":1249,"threads_started":1}
I20260806 09:33:46.009473 17075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:45.987749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:40985 (local address 127.20.113.2:48133)
0806 09:33:45.989406 (+ 1657us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:45.989460 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 09:33:45.991295 (+ 1835us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:33:45.992047 (+ 752us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:33:45.992056 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:33:45.992770 (+ 714us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:33:45.993377 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:33:45.993395 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:33:45.994844 (+ 1449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:33:45.994849 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:33:45.996140 (+ 1291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:33:45.996158 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:33:45.997304 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:33:45.997309 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:33:45.997515 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:45.998299 (+ 784us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:33:45.998323 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:33:46.000707 (+ 2384us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:33:46.004847 (+ 4140us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:33:46.004853 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:33:46.004866 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:33:46.005315 (+ 449us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:33:46.005738 (+ 423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:33:46.005742 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:33:46.005744 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:33:46.005930 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:33:46.006563 (+ 633us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:33:46.006569 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:33:46.006888 (+ 319us) client_negotiation.cc:769] Sending connection context
0806 09:33:46.007231 (+ 343us) client_negotiation.cc:240] Negotiation successful
0806 09:33:46.007866 (+ 635us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1349,"thread_start_us":1075,"threads_started":1}
I20260806 09:33:46.011029 17072 heartbeater.cc:347] Connected to a master server at 127.20.113.62:40985
I20260806 09:33:46.011343 17072 heartbeater.cc:464] Registering TS with master...
I20260806 09:33:46.012096 17072 heartbeater.cc:511] Master 127.20.113.62:40985 requested a full tablet report, sending...
I20260806 09:33:46.013662 16593 ts_manager.cc:194] Registered new tserver with Master: 6c746c0cca474647a82aec2949649355 (127.20.113.2:41313)
I20260806 09:33:46.014513 16593 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.113.2@KRBTEST.COM'} at 127.20.113.2:48133
WARNING: no policy specified for kudu/127.20.113.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.113.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.113.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.3.keytab.
Entry for principal kudu/127.20.113.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.3.keytab.
I20260806 09:33:46.070281 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786008495763054-20932-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786008495763054-20932-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786008495763054-20932-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786008495763054-20932-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.113.3:0
--local_ip_for_outbound_sockets=127.20.113.3
--webserver_interface=127.20.113.3
--webserver_port=0
--tserver_master_addrs=127.20.113.62:40985
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.3.keytab
--principal=kudu/127.20.113.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:39385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:33:46.429868 17079 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:33:46.430225 17079 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:33:46.430392 17079 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:33:46.438400 17079 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:33:46.438558 17079 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:33:46.438732 17079 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.113.3
I20260806 09:33:46.448697 17079 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:39385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786008495763054-20932-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786008495763054-20932-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.3.keytab
--principal=kudu/127.20.113.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.113.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786008495763054-20932-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.113.3
--webserver_port=0
--tserver_master_addrs=127.20.113.62:40985
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.113.3
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786008495763054-20932-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:33:46.449935 17079 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:33:46.451284 17079 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:33:46.465023 17089 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:33:46.464983 17093 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:33:46.465065 17088 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:33:46.466109 17079 server_base.cc:1034] running on GCE node
I20260806 09:33:46.466724 17079 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:33:46.466912 16871 heartbeater.cc:503] Master 127.20.113.62:40985 was elected leader, sending a full tablet report...
I20260806 09:33:46.467844 17079 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:33:46.469089 17079 hybrid_clock.cc:648] HybridClock initialized: now 1786008826469029 us; error 52 us; skew 500 ppm
I20260806 09:33:46.469429 17079 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:33:46 dist-test-slave-45dx krb5kdc[16529](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008826, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:33:46.472754 17079 init.cc:377] Logged in from keytab as kudu/127.20.113.3@KRBTEST.COM (short username kudu)
I20260806 09:33:46.474315 17079 webserver.cc:460] Webserver started at http://127.20.113.3:42115/ using document root <none> and password file <none>
I20260806 09:33:46.475139 17079 fs_manager.cc:359] Metadata directory not provided
I20260806 09:33:46.475296 17079 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:33:46.475646 17079 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:33:46.478726 17079 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786008495763054-20932-0/minicluster-data/ts-2/data/instance:
uuid: "500773bb58a84d388d261611b5eccdec"
format_stamp: "Formatted at 2026-08-06 09:33:46 on dist-test-slave-45dx"
server_key: "cf805b10b111e74407fd43e22baa0169"
server_key_iv: "ef90dcf14854563d60f16c2c6b10f1f8"
server_key_version: "encryptionkey@0"
I20260806 09:33:46.479543 17079 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786008495763054-20932-0/minicluster-data/ts-2/wal/instance:
uuid: "500773bb58a84d388d261611b5eccdec"
format_stamp: "Formatted at 2026-08-06 09:33:46 on dist-test-slave-45dx"
server_key: "cf805b10b111e74407fd43e22baa0169"
server_key_iv: "ef90dcf14854563d60f16c2c6b10f1f8"
server_key_version: "encryptionkey@0"
I20260806 09:33:46.484982 17079 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 09:33:46.488494 17103 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:33:46.489777 17079 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 09:33:46.489956 17079 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786008495763054-20932-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786008495763054-20932-0/minicluster-data/ts-2/wal
uuid: "500773bb58a84d388d261611b5eccdec"
format_stamp: "Formatted at 2026-08-06 09:33:46 on dist-test-slave-45dx"
server_key: "cf805b10b111e74407fd43e22baa0169"
server_key_iv: "ef90dcf14854563d60f16c2c6b10f1f8"
server_key_version: "encryptionkey@0"
I20260806 09:33:46.490162 17079 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786008495763054-20932-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786008495763054-20932-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786008495763054-20932-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:33:46.510221 17079 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:33:46.515349 17079 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:33:46.515653 17079 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:33:46.516723 17079 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:33:46.518151 17079 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:33:46.518265 17079 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:33:46.518451 17079 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:33:46.518528 17079 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:33:46.552893 17079 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.3:34939
I20260806 09:33:46.552977 17270 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.3:34939 every 8 connection(s)
I20260806 09:33:46.554190 17079 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786008495763054-20932-0/minicluster-data/ts-2/data/info.pb
I20260806 09:33:46.557852 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 17079
I20260806 09:33:46.558055 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786008495763054-20932-0/minicluster-data/ts-2/wal/instance
I20260806 09:33:46.558657 20932 external_mini_cluster.cc:1428] Setting key e5aa713a9b3bcd6e2dd769c801802b43
Aug 06 09:33:46 dist-test-slave-45dx krb5kdc[16529](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008826, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.3@KRBTEST.COM for kudu/127.20.113.62@KRBTEST.COM
I20260806 09:33:46.574797 17273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:46.556768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.3:34859 (local address 127.20.113.62:40985)
0806 09:33:46.557239 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0806 09:33:46.557245 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:33:46.558484 (+ 1239us) server_negotiation.cc:407] Connection header received
0806 09:33:46.560989 (+ 2505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:33:46.560994 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:33:46.561077 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:33:46.561201 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:33:46.562776 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:46.563545 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:46.565114 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:46.565512 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:46.565656 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:46.569000 (+ 3344us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:33:46.569006 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:33:46.569012 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:33:46.569047 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:33:46.571502 (+ 2455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:33:46.572307 (+ 805us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:33:46.572314 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:33:46.572342 (+ 28us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:33:46.572480 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:33:46.573015 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:33:46.573020 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:33:46.573023 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:33:46.573369 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:33:46.573526 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0806 09:33:46.574318 (+ 792us) server_negotiation.cc:299] Negotiation successful
0806 09:33:46.574479 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":205,"threads_started":1}
I20260806 09:33:46.575765 17276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:46.557386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:40985 (local address 127.20.113.3:34859)
0806 09:33:46.558290 (+ 904us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:46.558353 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0806 09:33:46.560704 (+ 2351us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:33:46.561444 (+ 740us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:33:46.561451 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:33:46.562074 (+ 623us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:33:46.562606 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:33:46.562623 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:33:46.563687 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:33:46.563692 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:33:46.564964 (+ 1272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:33:46.564976 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:33:46.565743 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:33:46.565750 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:33:46.565978 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:46.566647 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:33:46.566672 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:33:46.568796 (+ 2124us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:33:46.571633 (+ 2837us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:33:46.571641 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:33:46.571654 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:33:46.572166 (+ 512us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:33:46.572595 (+ 429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:33:46.572600 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:33:46.572604 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:33:46.572862 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:33:46.573496 (+ 634us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:33:46.573502 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:33:46.573946 (+ 444us) client_negotiation.cc:769] Sending connection context
0806 09:33:46.574211 (+ 265us) client_negotiation.cc:240] Negotiation successful
0806 09:33:46.574623 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":593,"thread_start_us":342,"threads_started":1}
I20260806 09:33:46.577031 17271 heartbeater.cc:347] Connected to a master server at 127.20.113.62:40985
I20260806 09:33:46.577293 17271 heartbeater.cc:464] Registering TS with master...
I20260806 09:33:46.577947 17271 heartbeater.cc:511] Master 127.20.113.62:40985 requested a full tablet report, sending...
I20260806 09:33:46.579260 16593 ts_manager.cc:194] Registered new tserver with Master: 500773bb58a84d388d261611b5eccdec (127.20.113.3:34939)
I20260806 09:33:46.579975 16593 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.113.3@KRBTEST.COM'} at 127.20.113.3:34859
I20260806 09:33:46.585875 20932 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 09:33:46.602541 17273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:46.590150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60172 (local address 127.20.113.62:40985)
0806 09:33:46.590371 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0806 09:33:46.590380 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 09:33:46.590859 (+ 479us) server_negotiation.cc:407] Connection header received
0806 09:33:46.591003 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:33:46.591007 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:33:46.591068 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:33:46.591148 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:33:46.592617 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:46.593417 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:46.594850 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:46.595240 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:46.595330 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:46.597580 (+ 2250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:33:46.597585 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:33:46.597588 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:33:46.597618 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:33:46.600104 (+ 2486us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:33:46.600763 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:33:46.600767 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:33:46.600769 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:33:46.600888 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:33:46.601218 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:33:46.601221 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:33:46.601223 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:33:46.601517 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:33:46.601691 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0806 09:33:46.602020 (+ 329us) server_negotiation.cc:299] Negotiation successful
0806 09:33:46.602183 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 09:33:46.606153 16593 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60172:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 09:33:46.612883 16593 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 09:33:46.647279 17287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:46.624937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.2:41313 (local address 127.0.0.1:55168)
0806 09:33:46.637474 (+ 12537us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:46.637511 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 09:33:46.637728 (+ 217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:33:46.638504 (+ 776us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:33:46.638511 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:33:46.638543 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:33:46.638970 (+ 427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:33:46.638989 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:33:46.640969 (+ 1980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:33:46.640979 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:33:46.643901 (+ 2922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:33:46.643914 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:33:46.646213 (+ 2299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:33:46.646220 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:33:46.646444 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:46.646498 (+ 54us) client_negotiation.cc:769] Sending connection context
0806 09:33:46.646636 (+ 138us) client_negotiation.cc:240] Negotiation successful
0806 09:33:46.646830 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12313,"thread_start_us":12158,"threads_started":1}
I20260806 09:33:46.648375 17288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:46.625147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55168 (local address 127.20.113.2:41313)
0806 09:33:46.637033 (+ 11886us) server_negotiation.cc:206] Beginning negotiation
0806 09:33:46.637045 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 09:33:46.637922 (+ 877us) server_negotiation.cc:407] Connection header received
0806 09:33:46.638058 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:33:46.638066 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:33:46.638282 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:33:46.638481 (+ 199us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:33:46.639192 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:46.640792 (+ 1600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:46.644084 (+ 3292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:46.645629 (+ 1545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:46.645784 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:46.646032 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0806 09:33:46.647829 (+ 1797us) server_negotiation.cc:299] Negotiation successful
0806 09:33:46.648034 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11706,"thread_start_us":11594,"threads_started":1}
I20260806 09:33:46.648720 17293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:46.625274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.1:35793 (local address 127.0.0.1:48314)
0806 09:33:46.639805 (+ 14531us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:46.639822 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 09:33:46.639936 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:33:46.641218 (+ 1282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:33:46.641224 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:33:46.641244 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:33:46.641599 (+ 355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:33:46.641608 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:33:46.643436 (+ 1828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:33:46.643446 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:33:46.645614 (+ 2168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:33:46.645627 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:33:46.647919 (+ 2292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:33:46.647926 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:33:46.648105 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:46.648138 (+ 33us) client_negotiation.cc:769] Sending connection context
0806 09:33:46.648230 (+ 92us) client_negotiation.cc:240] Negotiation successful
0806 09:33:46.648398 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14404,"thread_start_us":16751,"threads_started":1}
I20260806 09:33:46.649603 17292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:46.625470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48314 (local address 127.20.113.1:35793)
0806 09:33:46.640480 (+ 15010us) server_negotiation.cc:206] Beginning negotiation
0806 09:33:46.640490 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 09:33:46.640517 (+ 27us) server_negotiation.cc:407] Connection header received
0806 09:33:46.640601 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:33:46.640608 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:33:46.640809 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:33:46.640998 (+ 189us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:33:46.641788 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:46.643211 (+ 1423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:46.645828 (+ 2617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:46.647393 (+ 1565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:46.647540 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:46.647784 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0806 09:33:46.649066 (+ 1282us) server_negotiation.cc:299] Negotiation successful
0806 09:33:46.649243 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14827,"thread_start_us":14976,"threads_started":1}
I20260806 09:33:46.650286 17289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:46.627237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.3:34939 (local address 127.0.0.1:44244)
0806 09:33:46.642358 (+ 15121us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:46.642376 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 09:33:46.642501 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:33:46.643053 (+ 552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:33:46.643058 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:33:46.643082 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:33:46.643579 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:33:46.643593 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:33:46.645752 (+ 2159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:33:46.645757 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:33:46.647359 (+ 1602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:33:46.647374 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:33:46.649261 (+ 1887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:33:46.649267 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:33:46.649472 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:46.649504 (+ 32us) client_negotiation.cc:769] Sending connection context
0806 09:33:46.649621 (+ 117us) client_negotiation.cc:240] Negotiation successful
0806 09:33:46.649909 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15001,"mutex_wait_us":1,"thread_start_us":12271,"threads_started":1}
I20260806 09:33:46.651883 16973 tablet_service.cc:1467] Processing CreateTablet for tablet 4f183e613441468a8f6b4d287ef901cf (DEFAULT_TABLE table=test-table [id=333e522692164fa98a9136a4c3f05124]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:33:46.652895 17294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:46.628040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44244 (local address 127.20.113.3:34939)
0806 09:33:46.639366 (+ 11326us) server_negotiation.cc:206] Beginning negotiation
0806 09:33:46.639374 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 09:33:46.642496 (+ 3122us) server_negotiation.cc:407] Connection header received
0806 09:33:46.642639 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:33:46.642647 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:33:46.642857 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:33:46.643039 (+ 182us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:33:46.644559 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:46.645578 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:46.647560 (+ 1982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:46.649054 (+ 1494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:46.649196 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:46.649429 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0806 09:33:46.649669 (+ 240us) server_negotiation.cc:299] Negotiation successful
0806 09:33:46.652532 (+ 2863us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11133,"thread_start_us":10975,"threads_started":1}
I20260806 09:33:46.652540 16772 tablet_service.cc:1467] Processing CreateTablet for tablet 4f183e613441468a8f6b4d287ef901cf (DEFAULT_TABLE table=test-table [id=333e522692164fa98a9136a4c3f05124]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:33:46.653403 16973 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f183e613441468a8f6b4d287ef901cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:33:46.654008 16772 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f183e613441468a8f6b4d287ef901cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:33:46.656462 17172 tablet_service.cc:1467] Processing CreateTablet for tablet 4f183e613441468a8f6b4d287ef901cf (DEFAULT_TABLE table=test-table [id=333e522692164fa98a9136a4c3f05124]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:33:46.657832 17172 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f183e613441468a8f6b4d287ef901cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:33:46.665158 17296 tablet_bootstrap.cc:492] T 4f183e613441468a8f6b4d287ef901cf P 8c95494edfe44953baf6c42b6c1a1e7e: Bootstrap starting.
I20260806 09:33:46.668766 17296 tablet_bootstrap.cc:654] T 4f183e613441468a8f6b4d287ef901cf P 8c95494edfe44953baf6c42b6c1a1e7e: Neither blocks nor log segments found. Creating new log.
I20260806 09:33:46.670082 17296 log.cc:824] T 4f183e613441468a8f6b4d287ef901cf P 8c95494edfe44953baf6c42b6c1a1e7e: Log is configured to *not* fsync() on all Append() calls
I20260806 09:33:46.672906 17296 tablet_bootstrap.cc:492] T 4f183e613441468a8f6b4d287ef901cf P 8c95494edfe44953baf6c42b6c1a1e7e: No bootstrap required, opened a new log
I20260806 09:33:46.672896 17301 tablet_bootstrap.cc:492] T 4f183e613441468a8f6b4d287ef901cf P 500773bb58a84d388d261611b5eccdec: Bootstrap starting.
I20260806 09:33:46.673535 17296 ts_tablet_manager.cc:1397] T 4f183e613441468a8f6b4d287ef901cf P 8c95494edfe44953baf6c42b6c1a1e7e: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260806 09:33:46.675904 17301 tablet_bootstrap.cc:654] T 4f183e613441468a8f6b4d287ef901cf P 500773bb58a84d388d261611b5eccdec: Neither blocks nor log segments found. Creating new log.
I20260806 09:33:46.676477 17298 tablet_bootstrap.cc:492] T 4f183e613441468a8f6b4d287ef901cf P 6c746c0cca474647a82aec2949649355: Bootstrap starting.
I20260806 09:33:46.677156 17301 log.cc:824] T 4f183e613441468a8f6b4d287ef901cf P 500773bb58a84d388d261611b5eccdec: Log is configured to *not* fsync() on all Append() calls
I20260806 09:33:46.679801 17298 tablet_bootstrap.cc:654] T 4f183e613441468a8f6b4d287ef901cf P 6c746c0cca474647a82aec2949649355: Neither blocks nor log segments found. Creating new log.
I20260806 09:33:46.680451 17301 tablet_bootstrap.cc:492] T 4f183e613441468a8f6b4d287ef901cf P 500773bb58a84d388d261611b5eccdec: No bootstrap required, opened a new log
I20260806 09:33:46.680724 17301 ts_tablet_manager.cc:1397] T 4f183e613441468a8f6b4d287ef901cf P 500773bb58a84d388d261611b5eccdec: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.005s
I20260806 09:33:46.681061 17298 log.cc:824] T 4f183e613441468a8f6b4d287ef901cf P 6c746c0cca474647a82aec2949649355: Log is configured to *not* fsync() on all Append() calls
I20260806 09:33:46.681725 17296 raft_consensus.cc:348] T 4f183e613441468a8f6b4d287ef901cf P 8c95494edfe44953baf6c42b6c1a1e7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c746c0cca474647a82aec2949649355" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 41313 } } peers { permanent_uuid: "8c95494edfe44953baf6c42b6c1a1e7e" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 35793 } } peers { permanent_uuid: "500773bb58a84d388d261611b5eccdec" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 34939 } }
I20260806 09:33:46.682250 17296 raft_consensus.cc:374] T 4f183e613441468a8f6b4d287ef901cf P 8c95494edfe44953baf6c42b6c1a1e7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:33:46.682379 17296 raft_consensus.cc:729] T 4f183e613441468a8f6b4d287ef901cf P 8c95494edfe44953baf6c42b6c1a1e7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c95494edfe44953baf6c42b6c1a1e7e, State: Initialized, Role: FOLLOWER
I20260806 09:33:46.683000 17296 consensus_queue.cc:260] T 4f183e613441468a8f6b4d287ef901cf P 8c95494edfe44953baf6c42b6c1a1e7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c746c0cca474647a82aec2949649355" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 41313 } } peers { permanent_uuid: "8c95494edfe44953baf6c42b6c1a1e7e" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 35793 } } peers { permanent_uuid: "500773bb58a84d388d261611b5eccdec" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 34939 } }
I20260806 09:33:46.684388 17298 tablet_bootstrap.cc:492] T 4f183e613441468a8f6b4d287ef901cf P 6c746c0cca474647a82aec2949649355: No bootstrap required, opened a new log
I20260806 09:33:46.684648 17298 ts_tablet_manager.cc:1397] T 4f183e613441468a8f6b4d287ef901cf P 6c746c0cca474647a82aec2949649355: Time spent bootstrapping tablet: real 0.013s user 0.007s sys 0.000s
I20260806 09:33:46.684705 17296 ts_tablet_manager.cc:1428] T 4f183e613441468a8f6b4d287ef901cf P 8c95494edfe44953baf6c42b6c1a1e7e: Time spent starting tablet: real 0.011s user 0.006s sys 0.004s
I20260806 09:33:46.691530 17301 raft_consensus.cc:348] T 4f183e613441468a8f6b4d287ef901cf P 500773bb58a84d388d261611b5eccdec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c746c0cca474647a82aec2949649355" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 41313 } } peers { permanent_uuid: "8c95494edfe44953baf6c42b6c1a1e7e" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 35793 } } peers { permanent_uuid: "500773bb58a84d388d261611b5eccdec" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 34939 } }
I20260806 09:33:46.692021 17301 raft_consensus.cc:374] T 4f183e613441468a8f6b4d287ef901cf P 500773bb58a84d388d261611b5eccdec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:33:46.692129 17301 raft_consensus.cc:729] T 4f183e613441468a8f6b4d287ef901cf P 500773bb58a84d388d261611b5eccdec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 500773bb58a84d388d261611b5eccdec, State: Initialized, Role: FOLLOWER
I20260806 09:33:46.692812 17301 consensus_queue.cc:260] T 4f183e613441468a8f6b4d287ef901cf P 500773bb58a84d388d261611b5eccdec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c746c0cca474647a82aec2949649355" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 41313 } } peers { permanent_uuid: "8c95494edfe44953baf6c42b6c1a1e7e" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 35793 } } peers { permanent_uuid: "500773bb58a84d388d261611b5eccdec" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 34939 } }
I20260806 09:33:46.694021 17271 heartbeater.cc:503] Master 127.20.113.62:40985 was elected leader, sending a full tablet report...
I20260806 09:33:46.694737 17301 ts_tablet_manager.cc:1428] T 4f183e613441468a8f6b4d287ef901cf P 500773bb58a84d388d261611b5eccdec: Time spent starting tablet: real 0.014s user 0.008s sys 0.005s
I20260806 09:33:46.695500 17298 raft_consensus.cc:348] T 4f183e613441468a8f6b4d287ef901cf P 6c746c0cca474647a82aec2949649355 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c746c0cca474647a82aec2949649355" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 41313 } } peers { permanent_uuid: "8c95494edfe44953baf6c42b6c1a1e7e" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 35793 } } peers { permanent_uuid: "500773bb58a84d388d261611b5eccdec" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 34939 } }
I20260806 09:33:46.695878 17298 raft_consensus.cc:374] T 4f183e613441468a8f6b4d287ef901cf P 6c746c0cca474647a82aec2949649355 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:33:46.695983 17298 raft_consensus.cc:729] T 4f183e613441468a8f6b4d287ef901cf P 6c746c0cca474647a82aec2949649355 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c746c0cca474647a82aec2949649355, State: Initialized, Role: FOLLOWER
I20260806 09:33:46.696646 17298 consensus_queue.cc:260] T 4f183e613441468a8f6b4d287ef901cf P 6c746c0cca474647a82aec2949649355 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c746c0cca474647a82aec2949649355" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 41313 } } peers { permanent_uuid: "8c95494edfe44953baf6c42b6c1a1e7e" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 35793 } } peers { permanent_uuid: "500773bb58a84d388d261611b5eccdec" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 34939 } }
I20260806 09:33:46.697800 17072 heartbeater.cc:503] Master 127.20.113.62:40985 was elected leader, sending a full tablet report...
I20260806 09:33:46.698385 17298 ts_tablet_manager.cc:1428] T 4f183e613441468a8f6b4d287ef901cf P 6c746c0cca474647a82aec2949649355: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
W20260806 09:33:46.736349 17073 tablet.cc:2369] T 4f183e613441468a8f6b4d287ef901cf P 6c746c0cca474647a82aec2949649355: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 09:33:46.758345 17306 raft_consensus.cc:482] T 4f183e613441468a8f6b4d287ef901cf P 500773bb58a84d388d261611b5eccdec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 09:33:46.758693 17306 raft_consensus.cc:504] T 4f183e613441468a8f6b4d287ef901cf P 500773bb58a84d388d261611b5eccdec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c746c0cca474647a82aec2949649355" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 41313 } } peers { permanent_uuid: "8c95494edfe44953baf6c42b6c1a1e7e" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 35793 } } peers { permanent_uuid: "500773bb58a84d388d261611b5eccdec" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 34939 } }
I20260806 09:33:46.760602 17306 leader_election.cc:288] T 4f183e613441468a8f6b4d287ef901cf P 500773bb58a84d388d261611b5eccdec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c746c0cca474647a82aec2949649355 (127.20.113.2:41313), 8c95494edfe44953baf6c42b6c1a1e7e (127.20.113.1:35793)
I20260806 09:33:46.766191 17276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:46.760836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.2:41313 (local address 127.20.113.3:40577)
0806 09:33:46.761121 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:46.761145 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 09:33:46.761278 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:33:46.761801 (+ 523us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:33:46.761807 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:33:46.761842 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:33:46.762010 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:33:46.762018 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:33:46.762998 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:33:46.763004 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:33:46.764375 (+ 1371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:33:46.764390 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:33:46.765464 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:33:46.765470 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:33:46.765659 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:46.765683 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 09:33:46.765786 (+ 103us) client_negotiation.cc:240] Negotiation successful
0806 09:33:46.765917 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95}
I20260806 09:33:46.766219 17288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:46.761193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.3:40577 (local address 127.20.113.2:41313)
0806 09:33:46.761431 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0806 09:33:46.761439 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 09:33:46.761463 (+ 24us) server_negotiation.cc:407] Connection header received
0806 09:33:46.761546 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:33:46.761554 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:33:46.761639 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:33:46.761737 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:33:46.762161 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:46.762864 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:46.764533 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:46.765295 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:46.765469 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:46.765543 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 09:33:46.765823 (+ 280us) server_negotiation.cc:299] Negotiation successful
0806 09:33:46.765980 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 09:33:46.767318 17310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:46.760965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.1:35793 (local address 127.20.113.3:37155)
0806 09:33:46.761959 (+ 994us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:46.761974 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 09:33:46.762112 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:33:46.762450 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:33:46.762454 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:33:46.762480 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:33:46.762815 (+ 335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:33:46.762824 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:33:46.763898 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:33:46.763902 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:33:46.765286 (+ 1384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:33:46.765298 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:33:46.766597 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:33:46.766602 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:33:46.766713 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:46.766734 (+ 21us) client_negotiation.cc:769] Sending connection context
0806 09:33:46.766835 (+ 101us) client_negotiation.cc:240] Negotiation successful
0806 09:33:46.767001 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":839,"thread_start_us":550,"threads_started":1}
I20260806 09:33:46.767313 17292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:46.761259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.3:37155 (local address 127.20.113.1:35793)
0806 09:33:46.761532 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0806 09:33:46.761541 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 09:33:46.762064 (+ 523us) server_negotiation.cc:407] Connection header received
0806 09:33:46.762264 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:33:46.762269 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:33:46.762329 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:33:46.762433 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:33:46.762976 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:46.763703 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:46.765461 (+ 1758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:46.766345 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:46.766516 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:46.766609 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 09:33:46.766907 (+ 298us) server_negotiation.cc:299] Negotiation successful
0806 09:33:46.767056 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260806 09:33:46.767315 17304 raft_consensus.cc:482] T 4f183e613441468a8f6b4d287ef901cf P 8c95494edfe44953baf6c42b6c1a1e7e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 09:33:46.767360 17001 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f183e613441468a8f6b4d287ef901cf" candidate_uuid: "500773bb58a84d388d261611b5eccdec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c746c0cca474647a82aec2949649355" is_pre_election: true
I20260806 09:33:46.767989 17001 raft_consensus.cc:2454] T 4f183e613441468a8f6b4d287ef901cf P 6c746c0cca474647a82aec2949649355 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 500773bb58a84d388d261611b5eccdec in term 0.
I20260806 09:33:46.767810 17304 raft_consensus.cc:504] T 4f183e613441468a8f6b4d287ef901cf P 8c95494edfe44953baf6c42b6c1a1e7e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c746c0cca474647a82aec2949649355" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 41313 } } peers { permanent_uuid: "8c95494edfe44953baf6c42b6c1a1e7e" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 35793 } } peers { permanent_uuid: "500773bb58a84d388d261611b5eccdec" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 34939 } }
I20260806 09:33:46.768705 17108 leader_election.cc:302] T 4f183e613441468a8f6b4d287ef901cf P 500773bb58a84d388d261611b5eccdec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 500773bb58a84d388d261611b5eccdec, 6c746c0cca474647a82aec2949649355; no voters:
I20260806 09:33:46.768631 16802 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f183e613441468a8f6b4d287ef901cf" candidate_uuid: "500773bb58a84d388d261611b5eccdec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c95494edfe44953baf6c42b6c1a1e7e" is_pre_election: true
I20260806 09:33:46.769102 17306 raft_consensus.cc:2781] T 4f183e613441468a8f6b4d287ef901cf P 500773bb58a84d388d261611b5eccdec [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 09:33:46.769100 16802 raft_consensus.cc:2454] T 4f183e613441468a8f6b4d287ef901cf P 8c95494edfe44953baf6c42b6c1a1e7e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 500773bb58a84d388d261611b5eccdec in term 0.
I20260806 09:33:46.769249 17306 raft_consensus.cc:482] T 4f183e613441468a8f6b4d287ef901cf P 500773bb58a84d388d261611b5eccdec [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 09:33:46.769384 17306 raft_consensus.cc:3037] T 4f183e613441468a8f6b4d287ef901cf P 500773bb58a84d388d261611b5eccdec [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:33:46.772444 17304 leader_election.cc:288] T 4f183e613441468a8f6b4d287ef901cf P 8c95494edfe44953baf6c42b6c1a1e7e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c746c0cca474647a82aec2949649355 (127.20.113.2:41313), 500773bb58a84d388d261611b5eccdec (127.20.113.3:34939)
I20260806 09:33:46.772563 17306 raft_consensus.cc:504] T 4f183e613441468a8f6b4d287ef901cf P 500773bb58a84d388d261611b5eccdec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c746c0cca474647a82aec2949649355" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 41313 } } peers { permanent_uuid: "8c95494edfe44953baf6c42b6c1a1e7e" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 35793 } } peers { permanent_uuid: "500773bb58a84d388d261611b5eccdec" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 34939 } }
I20260806 09:33:46.773594 17001 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f183e613441468a8f6b4d287ef901cf" candidate_uuid: "500773bb58a84d388d261611b5eccdec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c746c0cca474647a82aec2949649355"
I20260806 09:33:46.773798 17001 raft_consensus.cc:3037] T 4f183e613441468a8f6b4d287ef901cf P 6c746c0cca474647a82aec2949649355 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:33:46.777199 17001 raft_consensus.cc:2454] T 4f183e613441468a8f6b4d287ef901cf P 6c746c0cca474647a82aec2949649355 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 500773bb58a84d388d261611b5eccdec in term 1.
I20260806 09:33:46.777671 17108 leader_election.cc:302] T 4f183e613441468a8f6b4d287ef901cf P 500773bb58a84d388d261611b5eccdec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 500773bb58a84d388d261611b5eccdec, 6c746c0cca474647a82aec2949649355; no voters:
I20260806 09:33:46.780267 17288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:46.770120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.1:39697 (local address 127.20.113.2:41313)
0806 09:33:46.770290 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0806 09:33:46.770297 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:33:46.771013 (+ 716us) server_negotiation.cc:407] Connection header received
0806 09:33:46.773287 (+ 2274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:33:46.773293 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:33:46.773356 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:33:46.773409 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:33:46.774073 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:46.775263 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:46.777936 (+ 2673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:46.779133 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:46.779257 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:46.779322 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 09:33:46.779870 (+ 548us) server_negotiation.cc:299] Negotiation successful
0806 09:33:46.779998 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 09:33:46.781002 17312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:46.770056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.2:41313 (local address 127.20.113.1:39697)
0806 09:33:46.770864 (+ 808us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:46.770886 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 09:33:46.773138 (+ 2252us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:33:46.773566 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:33:46.773570 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:33:46.773598 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:33:46.773930 (+ 332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:33:46.773940 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:33:46.775468 (+ 1528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:33:46.775473 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:33:46.777752 (+ 2279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:33:46.777768 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:33:46.779502 (+ 1734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:33:46.779508 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:33:46.779674 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:46.779715 (+ 41us) client_negotiation.cc:769] Sending connection context
0806 09:33:46.780482 (+ 767us) client_negotiation.cc:240] Negotiation successful
0806 09:33:46.780615 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":652,"thread_start_us":415,"threads_started":1}
I20260806 09:33:46.781041 17306 leader_election.cc:288] T 4f183e613441468a8f6b4d287ef901cf P 500773bb58a84d388d261611b5eccdec [CANDIDATE]: Term 1 election: Requested vote from peers 6c746c0cca474647a82aec2949649355 (127.20.113.2:41313), 8c95494edfe44953baf6c42b6c1a1e7e (127.20.113.1:35793)
I20260806 09:33:46.781246 17306 raft_consensus.cc:2781] T 4f183e613441468a8f6b4d287ef901cf P 500773bb58a84d388d261611b5eccdec [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:33:46.781389 17306 raft_consensus.cc:686] T 4f183e613441468a8f6b4d287ef901cf P 500773bb58a84d388d261611b5eccdec [term 1 LEADER]: Becoming Leader. State: Replica: 500773bb58a84d388d261611b5eccdec, State: Running, Role: LEADER
I20260806 09:33:46.781510 17001 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f183e613441468a8f6b4d287ef901cf" candidate_uuid: "8c95494edfe44953baf6c42b6c1a1e7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c746c0cca474647a82aec2949649355" is_pre_election: true
I20260806 09:33:46.781529 16802 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f183e613441468a8f6b4d287ef901cf" candidate_uuid: "500773bb58a84d388d261611b5eccdec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c95494edfe44953baf6c42b6c1a1e7e"
I20260806 09:33:46.781718 16802 raft_consensus.cc:3037] T 4f183e613441468a8f6b4d287ef901cf P 8c95494edfe44953baf6c42b6c1a1e7e [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:33:46.781852 17001 raft_consensus.cc:2379] T 4f183e613441468a8f6b4d287ef901cf P 6c746c0cca474647a82aec2949649355 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c95494edfe44953baf6c42b6c1a1e7e in current term 1: Already voted for candidate 500773bb58a84d388d261611b5eccdec in this term.
I20260806 09:33:46.781993 17294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:46.770182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.1:37739 (local address 127.20.113.3:34939)
0806 09:33:46.770406 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0806 09:33:46.770414 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 09:33:46.773265 (+ 2851us) server_negotiation.cc:407] Connection header received
0806 09:33:46.773529 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:33:46.773534 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:33:46.773606 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:33:46.773691 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:33:46.774595 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:46.775616 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:46.778444 (+ 2828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:46.780147 (+ 1703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:46.780530 (+ 383us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:46.780646 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0806 09:33:46.781609 (+ 963us) server_negotiation.cc:299] Negotiation successful
0806 09:33:46.781730 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 09:33:46.783186 17313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:46.770162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.3:34939 (local address 127.20.113.1:37739)
0806 09:33:46.773180 (+ 3018us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:46.773199 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 09:33:46.773375 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:33:46.774001 (+ 626us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:33:46.774008 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:33:46.774033 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:33:46.774412 (+ 379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:33:46.774425 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:33:46.775842 (+ 1417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:33:46.775847 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:33:46.778196 (+ 2349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:33:46.778217 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:33:46.780901 (+ 2684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:33:46.780909 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:33:46.781288 (+ 379us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:46.781401 (+ 113us) client_negotiation.cc:769] Sending connection context
0806 09:33:46.782366 (+ 965us) client_negotiation.cc:240] Negotiation successful
0806 09:33:46.782594 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2894,"mutex_wait_us":5,"thread_start_us":2718,"threads_started":1}
I20260806 09:33:46.781965 17306 consensus_queue.cc:237] T 4f183e613441468a8f6b4d287ef901cf P 500773bb58a84d388d261611b5eccdec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c746c0cca474647a82aec2949649355" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 41313 } } peers { permanent_uuid: "8c95494edfe44953baf6c42b6c1a1e7e" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 35793 } } peers { permanent_uuid: "500773bb58a84d388d261611b5eccdec" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 34939 } }
I20260806 09:33:46.784374 17203 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f183e613441468a8f6b4d287ef901cf" candidate_uuid: "8c95494edfe44953baf6c42b6c1a1e7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "500773bb58a84d388d261611b5eccdec" is_pre_election: true
I20260806 09:33:46.785142 16802 raft_consensus.cc:2454] T 4f183e613441468a8f6b4d287ef901cf P 8c95494edfe44953baf6c42b6c1a1e7e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 500773bb58a84d388d261611b5eccdec in term 1.
I20260806 09:33:46.785501 16713 leader_election.cc:302] T 4f183e613441468a8f6b4d287ef901cf P 8c95494edfe44953baf6c42b6c1a1e7e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c95494edfe44953baf6c42b6c1a1e7e; no voters: 500773bb58a84d388d261611b5eccdec, 6c746c0cca474647a82aec2949649355
I20260806 09:33:46.785873 17304 raft_consensus.cc:2726] T 4f183e613441468a8f6b4d287ef901cf P 8c95494edfe44953baf6c42b6c1a1e7e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 09:33:46.793809 16593 catalog_manager.cc:5495] T 4f183e613441468a8f6b4d287ef901cf P 500773bb58a84d388d261611b5eccdec reported cstate change: term changed from 0 to 1, leader changed from <none> to 500773bb58a84d388d261611b5eccdec (127.20.113.3). New cstate: current_term: 1 leader_uuid: "500773bb58a84d388d261611b5eccdec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c746c0cca474647a82aec2949649355" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 41313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c95494edfe44953baf6c42b6c1a1e7e" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 35793 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "500773bb58a84d388d261611b5eccdec" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 34939 } health_report { overall_health: HEALTHY } } }
W20260806 09:33:46.805766 17272 tablet.cc:2369] T 4f183e613441468a8f6b4d287ef901cf P 500773bb58a84d388d261611b5eccdec: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 09:33:46.839076 17294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:46.833426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44248 (local address 127.20.113.3:34939)
0806 09:33:46.833632 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0806 09:33:46.833637 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:33:46.833658 (+ 21us) server_negotiation.cc:407] Connection header received
0806 09:33:46.833929 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:33:46.833933 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:33:46.834044 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:33:46.834213 (+ 169us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 09:33:46.834613 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:46.835403 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:46.836881 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:46.837293 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:46.837448 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:46.837950 (+ 502us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 09:33:46.838460 (+ 510us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 09:33:46.838624 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0806 09:33:46.838685 (+ 61us) server_negotiation.cc:299] Negotiation successful
0806 09:33:46.838767 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 09:33:46.847605 17001 raft_consensus.cc:1261] T 4f183e613441468a8f6b4d287ef901cf P 6c746c0cca474647a82aec2949649355 [term 1 FOLLOWER]: Refusing update from remote peer 500773bb58a84d388d261611b5eccdec: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 09:33:46.847648 16802 raft_consensus.cc:1261] T 4f183e613441468a8f6b4d287ef901cf P 8c95494edfe44953baf6c42b6c1a1e7e [term 1 FOLLOWER]: Refusing update from remote peer 500773bb58a84d388d261611b5eccdec: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 09:33:46.848580 17314 consensus_queue.cc:1035] T 4f183e613441468a8f6b4d287ef901cf P 500773bb58a84d388d261611b5eccdec [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c746c0cca474647a82aec2949649355" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 41313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 09:33:46.848923 17306 consensus_queue.cc:1035] T 4f183e613441468a8f6b4d287ef901cf P 500773bb58a84d388d261611b5eccdec [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c95494edfe44953baf6c42b6c1a1e7e" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 35793 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 09:33:46.860437 17318 mvcc.cc:205] Tried to move back new op lower bound from 7315492154757611520 to 7315492154518798336. Current Snapshot: MvccSnapshot[applied={T|T < 7315492154757611520}]
I20260806 09:33:46.861020 17320 mvcc.cc:205] Tried to move back new op lower bound from 7315492154757611520 to 7315492154518798336. Current Snapshot: MvccSnapshot[applied={T|T < 7315492154757611520}]
I20260806 09:33:46.874157 16590 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60172:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 09:33:46.874518 16590 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60172:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 09:33:46.878584 16590 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 932b787c8f99437bab3c970e2b7689db: Sending DeleteTablet for 3 replicas of tablet 4f183e613441468a8f6b4d287ef901cf
I20260806 09:33:46.879733 16772 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f183e613441468a8f6b4d287ef901cf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:33:46 UTC) from {username='kudu', principal='kudu/127.20.113.62@KRBTEST.COM'} at 127.0.0.1:48314
I20260806 09:33:46.879746 16973 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f183e613441468a8f6b4d287ef901cf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:33:46 UTC) from {username='kudu', principal='kudu/127.20.113.62@KRBTEST.COM'} at 127.0.0.1:55168
I20260806 09:33:46.880749 17172 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f183e613441468a8f6b4d287ef901cf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:33:46 UTC) from {username='kudu', principal='kudu/127.20.113.62@KRBTEST.COM'} at 127.0.0.1:44244
I20260806 09:33:46.881242 17334 tablet_replica.cc:330] T 4f183e613441468a8f6b4d287ef901cf P 6c746c0cca474647a82aec2949649355: stopping tablet replica
I20260806 09:33:46.881662 17334 raft_consensus.cc:2229] T 4f183e613441468a8f6b4d287ef901cf P 6c746c0cca474647a82aec2949649355 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 09:33:46.882125 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 16686
I20260806 09:33:46.882200 17334 raft_consensus.cc:2258] T 4f183e613441468a8f6b4d287ef901cf P 6c746c0cca474647a82aec2949649355 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 09:33:46.888396 17333 tablet_replica.cc:330] T 4f183e613441468a8f6b4d287ef901cf P 8c95494edfe44953baf6c42b6c1a1e7e: stopping tablet replica
I20260806 09:33:46.889422 17333 raft_consensus.cc:2229] T 4f183e613441468a8f6b4d287ef901cf P 8c95494edfe44953baf6c42b6c1a1e7e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 09:33:46.889951 17333 raft_consensus.cc:2258] T 4f183e613441468a8f6b4d287ef901cf P 8c95494edfe44953baf6c42b6c1a1e7e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 09:33:46.890815 17338 tablet_replica.cc:330] T 4f183e613441468a8f6b4d287ef901cf P 500773bb58a84d388d261611b5eccdec: stopping tablet replica
I20260806 09:33:46.891247 17338 raft_consensus.cc:2229] T 4f183e613441468a8f6b4d287ef901cf P 500773bb58a84d388d261611b5eccdec [term 1 LEADER]: Raft consensus shutting down.
I20260806 09:33:46.891963 17338 raft_consensus.cc:2258] T 4f183e613441468a8f6b4d287ef901cf P 500773bb58a84d388d261611b5eccdec [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 09:33:46.892161 17334 ts_tablet_manager.cc:1905] T 4f183e613441468a8f6b4d287ef901cf P 6c746c0cca474647a82aec2949649355: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 09:33:46.893883 17333 ts_tablet_manager.cc:1905] T 4f183e613441468a8f6b4d287ef901cf P 8c95494edfe44953baf6c42b6c1a1e7e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 09:33:46.894170 17338 ts_tablet_manager.cc:1905] T 4f183e613441468a8f6b4d287ef901cf P 500773bb58a84d388d261611b5eccdec: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 06 09:33:46 dist-test-slave-45dx krb5kdc[16529](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008824, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.113.1@KRBTEST.COM
I20260806 09:33:46.902163 17338 ts_tablet_manager.cc:1918] T 4f183e613441468a8f6b4d287ef901cf P 500773bb58a84d388d261611b5eccdec: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 09:33:46.902427 17338 log.cc:1196] T 4f183e613441468a8f6b4d287ef901cf P 500773bb58a84d388d261611b5eccdec: Deleting WAL directory at /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786008495763054-20932-0/minicluster-data/ts-2/wal/wals/4f183e613441468a8f6b4d287ef901cf
I20260806 09:33:46.903098 17338 ts_tablet_manager.cc:1939] T 4f183e613441468a8f6b4d287ef901cf P 500773bb58a84d388d261611b5eccdec: Deleting consensus metadata
I20260806 09:33:46.905203 16576 catalog_manager.cc:4841] TS 500773bb58a84d388d261611b5eccdec (127.20.113.3:34939): tablet 4f183e613441468a8f6b4d287ef901cf (table test-table [id=333e522692164fa98a9136a4c3f05124]) successfully deleted
I20260806 09:33:46.906589 17334 ts_tablet_manager.cc:1918] T 4f183e613441468a8f6b4d287ef901cf P 6c746c0cca474647a82aec2949649355: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 09:33:46.906814 17334 log.cc:1196] T 4f183e613441468a8f6b4d287ef901cf P 6c746c0cca474647a82aec2949649355: Deleting WAL directory at /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786008495763054-20932-0/minicluster-data/ts-1/wal/wals/4f183e613441468a8f6b4d287ef901cf
I20260806 09:33:46.907462 17334 ts_tablet_manager.cc:1939] T 4f183e613441468a8f6b4d287ef901cf P 6c746c0cca474647a82aec2949649355: Deleting consensus metadata
I20260806 09:33:46.908538 17333 ts_tablet_manager.cc:1918] T 4f183e613441468a8f6b4d287ef901cf P 8c95494edfe44953baf6c42b6c1a1e7e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 09:33:46.909057 17333 log.cc:1196] T 4f183e613441468a8f6b4d287ef901cf P 8c95494edfe44953baf6c42b6c1a1e7e: Deleting WAL directory at /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786008495763054-20932-0/minicluster-data/ts-0/wal/wals/4f183e613441468a8f6b4d287ef901cf
I20260806 09:33:46.909634 16575 catalog_manager.cc:4841] TS 6c746c0cca474647a82aec2949649355 (127.20.113.2:41313): tablet 4f183e613441468a8f6b4d287ef901cf (table test-table [id=333e522692164fa98a9136a4c3f05124]) successfully deleted
I20260806 09:33:46.909718 17333 ts_tablet_manager.cc:1939] T 4f183e613441468a8f6b4d287ef901cf P 8c95494edfe44953baf6c42b6c1a1e7e: Deleting consensus metadata
I20260806 09:33:46.911370 17292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:46.882953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48324 (local address 127.20.113.1:35793)
0806 09:33:46.883131 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0806 09:33:46.883140 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 09:33:46.889208 (+ 6068us) server_negotiation.cc:407] Connection header received
0806 09:33:46.890095 (+ 887us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:33:46.890102 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:33:46.890210 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:33:46.890287 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:33:46.892289 (+ 2002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:46.893433 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:46.895425 (+ 1992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:46.896129 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:46.896232 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:46.902794 (+ 6562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:33:46.902813 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:33:46.902837 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:33:46.902894 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:33:46.907333 (+ 4439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:33:46.908452 (+ 1119us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:33:46.908464 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:33:46.908469 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:33:46.908712 (+ 243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:33:46.909226 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:33:46.909231 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:33:46.909234 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:33:46.909789 (+ 555us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:33:46.910104 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0806 09:33:46.910654 (+ 550us) server_negotiation.cc:299] Negotiation successful
0806 09:33:46.910929 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 09:33:46.911692 16573 catalog_manager.cc:4841] TS 8c95494edfe44953baf6c42b6c1a1e7e (127.20.113.1:35793): tablet 4f183e613441468a8f6b4d287ef901cf (table test-table [id=333e522692164fa98a9136a4c3f05124]) successfully deleted
I20260806 09:33:46.912103 16865 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:33:47.069473 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 16686
I20260806 09:33:47.095816 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 16885
Aug 06 09:33:47 dist-test-slave-45dx krb5kdc[16529](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008824, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.113.2@KRBTEST.COM
I20260806 09:33:47.113698 17288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:47.096638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55182 (local address 127.20.113.2:41313)
0806 09:33:47.096814 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0806 09:33:47.096820 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:33:47.096990 (+ 170us) server_negotiation.cc:407] Connection header received
0806 09:33:47.097173 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:33:47.097180 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:33:47.097299 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:33:47.097416 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:33:47.098960 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:47.100128 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:47.101620 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:47.102286 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:47.102420 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:47.106851 (+ 4431us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:33:47.106873 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:33:47.106896 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:33:47.106943 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:33:47.110676 (+ 3733us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:33:47.111299 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:33:47.111308 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:33:47.111312 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:33:47.111466 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:33:47.111934 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:33:47.111940 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:33:47.111943 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:33:47.112466 (+ 523us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:33:47.112827 (+ 361us) server_negotiation.cc:1035] Waiting for connection context
0806 09:33:47.113095 (+ 268us) server_negotiation.cc:299] Negotiation successful
0806 09:33:47.113324 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 09:33:47.114466 17063 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:33:47.248579 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 16885
I20260806 09:33:47.280813 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 17079
Aug 06 09:33:47 dist-test-slave-45dx krb5kdc[16529](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008824, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.113.3@KRBTEST.COM
I20260806 09:33:47.299501 17294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:47.281592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44264 (local address 127.20.113.3:34939)
0806 09:33:47.281805 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0806 09:33:47.281809 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 09:33:47.281885 (+ 76us) server_negotiation.cc:407] Connection header received
0806 09:33:47.282093 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:33:47.282098 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:33:47.282159 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:33:47.282239 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:33:47.284164 (+ 1925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:47.285131 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:47.287005 (+ 1874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:47.287460 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:47.287557 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:47.293185 (+ 5628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:33:47.293209 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:33:47.293225 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:33:47.293272 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:33:47.296645 (+ 3373us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:33:47.297297 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:33:47.297305 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:33:47.297307 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:33:47.297439 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:33:47.297870 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:33:47.297877 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:33:47.297884 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:33:47.298288 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:33:47.298639 (+ 351us) server_negotiation.cc:1035] Waiting for connection context
0806 09:33:47.298924 (+ 285us) server_negotiation.cc:299] Negotiation successful
0806 09:33:47.299163 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 09:33:47.300153 17262 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:33:47.423166 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 17079
I20260806 09:33:47.453316 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 16547
I20260806 09:33:47.454434 16638 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:33:47.563987 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 16547
2026-08-06T09:33:47Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (4896 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-06T09:33:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T09:33:47Z Disabled control of system clock
I20260806 09:33:47.670298 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786008495763054-20932-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786008495763054-20932-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786008495763054-20932-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.113.62:42369
--webserver_interface=127.20.113.62
--webserver_port=0
--builtin_ntp_servers=127.20.113.20:41597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.113.62:42369
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 09:33:48.012893 17350 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:33:48.013209 17350 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:33:48.013293 17350 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:33:48.020334 17350 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 09:33:48.020433 17350 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:33:48.020469 17350 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:33:48.020524 17350 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 09:33:48.020566 17350 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 09:33:48.029594 17350 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:41597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786008495763054-20932-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786008495763054-20932-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.113.62:42369
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.113.62:42369
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.113.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786008495763054-20932-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:33:48.030853 17350 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:33:48.032582 17350 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:33:48.044409 17359 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:33:48.044368 17364 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:33:48.044446 17360 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:33:48.045656 17350 server_base.cc:1034] running on GCE node
I20260806 09:33:48.046392 17350 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:33:48.047658 17350 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:33:48.048872 17350 hybrid_clock.cc:648] HybridClock initialized: now 1786008828048818 us; error 30 us; skew 500 ppm
I20260806 09:33:48.049172 17350 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 09:33:48.051403 17350 webserver.cc:460] Webserver started at http://127.20.113.62:37757/ using document root <none> and password file <none>
I20260806 09:33:48.052028 17350 fs_manager.cc:359] Metadata directory not provided
I20260806 09:33:48.052114 17350 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:33:48.052470 17350 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:33:48.055233 17350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786008495763054-20932-0/minicluster-data/master-0/data/instance:
uuid: "f7046608bce1422e970b014cd0ee9cd4"
format_stamp: "Formatted at 2026-08-06 09:33:48 on dist-test-slave-45dx"
server_key: "35e83d4716d7f94b3ef68e7a1e518449"
server_key_iv: "2c8891e05bc9d3412cce804a679ea0ed"
server_key_version: "encryptionkey@0"
I20260806 09:33:48.056025 17350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786008495763054-20932-0/minicluster-data/master-0/wal/instance:
uuid: "f7046608bce1422e970b014cd0ee9cd4"
format_stamp: "Formatted at 2026-08-06 09:33:48 on dist-test-slave-45dx"
server_key: "35e83d4716d7f94b3ef68e7a1e518449"
server_key_iv: "2c8891e05bc9d3412cce804a679ea0ed"
server_key_version: "encryptionkey@0"
I20260806 09:33:48.060913 17350 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 09:33:48.064255 17376 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:33:48.065559 17350 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 09:33:48.065703 17350 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786008495763054-20932-0/minicluster-data/master-0/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786008495763054-20932-0/minicluster-data/master-0/wal
uuid: "f7046608bce1422e970b014cd0ee9cd4"
format_stamp: "Formatted at 2026-08-06 09:33:48 on dist-test-slave-45dx"
server_key: "35e83d4716d7f94b3ef68e7a1e518449"
server_key_iv: "2c8891e05bc9d3412cce804a679ea0ed"
server_key_version: "encryptionkey@0"
I20260806 09:33:48.065882 17350 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786008495763054-20932-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786008495763054-20932-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786008495763054-20932-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:33:48.081971 17350 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:33:48.082653 17350 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:33:48.082891 17350 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:33:48.101154 17350 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.62:42369
I20260806 09:33:48.101191 17456 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.62:42369 every 8 connection(s)
I20260806 09:33:48.102447 17350 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
I20260806 09:33:48.106050 17457 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:33:48.110297 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 17350
I20260806 09:33:48.110498 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786008495763054-20932-0/minicluster-data/master-0/wal/instance
I20260806 09:33:48.111064 20932 external_mini_cluster.cc:1428] Setting key 1fc2176d3cfdd36114dca450347bae63
I20260806 09:33:48.117452 17457 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f7046608bce1422e970b014cd0ee9cd4: Bootstrap starting.
I20260806 09:33:48.120930 17457 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f7046608bce1422e970b014cd0ee9cd4: Neither blocks nor log segments found. Creating new log.
I20260806 09:33:48.122153 17457 log.cc:824] T 00000000000000000000000000000000 P f7046608bce1422e970b014cd0ee9cd4: Log is configured to *not* fsync() on all Append() calls
I20260806 09:33:48.125288 17457 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f7046608bce1422e970b014cd0ee9cd4: No bootstrap required, opened a new log
I20260806 09:33:48.125311 17461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:48.112916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44280 (local address 127.20.113.62:42369)
0806 09:33:48.113791 (+ 875us) server_negotiation.cc:206] Beginning negotiation
0806 09:33:48.113800 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 09:33:48.113838 (+ 38us) server_negotiation.cc:407] Connection header received
0806 09:33:48.115857 (+ 2019us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:33:48.115877 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:33:48.116254 (+ 377us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:33:48.116603 (+ 349us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:33:48.117171 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:48.118488 (+ 1317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:48.120400 (+ 1912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:48.121388 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:48.121583 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:48.122954 (+ 1371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:33:48.122965 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:33:48.122987 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:33:48.123708 (+ 721us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:33:48.123782 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:33:48.123806 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:33:48.123895 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 09:33:48.124121 (+ 226us) server_negotiation.cc:299] Negotiation successful
0806 09:33:48.124449 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":540,"thread_start_us":341,"threads_started":1}
I20260806 09:33:48.132987 17457 raft_consensus.cc:348] T 00000000000000000000000000000000 P f7046608bce1422e970b014cd0ee9cd4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7046608bce1422e970b014cd0ee9cd4" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 42369 } }
I20260806 09:33:48.133426 17457 raft_consensus.cc:374] T 00000000000000000000000000000000 P f7046608bce1422e970b014cd0ee9cd4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:33:48.133538 17457 raft_consensus.cc:729] T 00000000000000000000000000000000 P f7046608bce1422e970b014cd0ee9cd4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7046608bce1422e970b014cd0ee9cd4, State: Initialized, Role: FOLLOWER
I20260806 09:33:48.134048 17457 consensus_queue.cc:260] T 00000000000000000000000000000000 P f7046608bce1422e970b014cd0ee9cd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7046608bce1422e970b014cd0ee9cd4" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 42369 } }
I20260806 09:33:48.134280 17457 raft_consensus.cc:388] T 00000000000000000000000000000000 P f7046608bce1422e970b014cd0ee9cd4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 09:33:48.134438 17457 raft_consensus.cc:482] T 00000000000000000000000000000000 P f7046608bce1422e970b014cd0ee9cd4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 09:33:48.134621 17457 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f7046608bce1422e970b014cd0ee9cd4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:33:48.136863 17457 raft_consensus.cc:504] T 00000000000000000000000000000000 P f7046608bce1422e970b014cd0ee9cd4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7046608bce1422e970b014cd0ee9cd4" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 42369 } }
I20260806 09:33:48.137430 17457 leader_election.cc:302] T 00000000000000000000000000000000 P f7046608bce1422e970b014cd0ee9cd4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f7046608bce1422e970b014cd0ee9cd4; no voters:
I20260806 09:33:48.138025 17457 leader_election.cc:288] T 00000000000000000000000000000000 P f7046608bce1422e970b014cd0ee9cd4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 09:33:48.138164 17463 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f7046608bce1422e970b014cd0ee9cd4 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:33:48.138868 17463 raft_consensus.cc:686] T 00000000000000000000000000000000 P f7046608bce1422e970b014cd0ee9cd4 [term 1 LEADER]: Becoming Leader. State: Replica: f7046608bce1422e970b014cd0ee9cd4, State: Running, Role: LEADER
I20260806 09:33:48.139389 17463 consensus_queue.cc:237] T 00000000000000000000000000000000 P f7046608bce1422e970b014cd0ee9cd4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7046608bce1422e970b014cd0ee9cd4" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 42369 } }
I20260806 09:33:48.140043 17457 sys_catalog.cc:564] T 00000000000000000000000000000000 P f7046608bce1422e970b014cd0ee9cd4 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 09:33:48.142030 17464 sys_catalog.cc:455] T 00000000000000000000000000000000 P f7046608bce1422e970b014cd0ee9cd4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f7046608bce1422e970b014cd0ee9cd4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7046608bce1422e970b014cd0ee9cd4" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 42369 } } }
I20260806 09:33:48.142546 17464 sys_catalog.cc:458] T 00000000000000000000000000000000 P f7046608bce1422e970b014cd0ee9cd4 [sys.catalog]: This master's current role is: LEADER
I20260806 09:33:48.142551 17466 sys_catalog.cc:455] T 00000000000000000000000000000000 P f7046608bce1422e970b014cd0ee9cd4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f7046608bce1422e970b014cd0ee9cd4. Latest consensus state: current_term: 1 leader_uuid: "f7046608bce1422e970b014cd0ee9cd4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7046608bce1422e970b014cd0ee9cd4" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 42369 } } }
I20260806 09:33:48.142987 17466 sys_catalog.cc:458] T 00000000000000000000000000000000 P f7046608bce1422e970b014cd0ee9cd4 [sys.catalog]: This master's current role is: LEADER
I20260806 09:33:48.143828 17474 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 09:33:48.149101 17474 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 09:33:48.157145 17474 catalog_manager.cc:1347] Generated new cluster ID: 80756374b9704002aac561153e345be4
I20260806 09:33:48.157248 17474 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 09:33:48.177346 17474 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 09:33:48.179076 17474 catalog_manager.cc:1504] Loading token signing keys...
I20260806 09:33:48.192107 17474 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f7046608bce1422e970b014cd0ee9cd4: Generated new TSK 0
I20260806 09:33:48.193080 17474 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 09:33:48.197254 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786008495763054-20932-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786008495763054-20932-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786008495763054-20932-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786008495763054-20932-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.113.1:0
--local_ip_for_outbound_sockets=127.20.113.1
--webserver_interface=127.20.113.1
--webserver_port=0
--tserver_master_addrs=127.20.113.62:42369
--builtin_ntp_servers=127.20.113.20:41597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 09:33:48.551366 17490 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:33:48.551668 17490 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:33:48.551852 17490 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:33:48.559029 17490 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:33:48.559168 17490 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:33:48.559361 17490 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.113.1
I20260806 09:33:48.568944 17490 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:41597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786008495763054-20932-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786008495763054-20932-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.113.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786008495763054-20932-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.113.1
--webserver_port=0
--tserver_master_addrs=127.20.113.62:42369
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.113.1
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786008495763054-20932-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:33:48.570372 17490 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:33:48.572209 17490 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:33:48.585875 17498 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:33:48.585876 17500 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:33:48.586019 17503 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:33:48.586591 17490 server_base.cc:1034] running on GCE node
I20260806 09:33:48.587338 17490 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:33:48.588254 17490 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:33:48.589502 17490 hybrid_clock.cc:648] HybridClock initialized: now 1786008828589449 us; error 64 us; skew 500 ppm
I20260806 09:33:48.589818 17490 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 09:33:48.591941 17490 webserver.cc:460] Webserver started at http://127.20.113.1:44897/ using document root <none> and password file <none>
I20260806 09:33:48.592700 17490 fs_manager.cc:359] Metadata directory not provided
I20260806 09:33:48.592826 17490 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:33:48.593140 17490 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:33:48.595881 17490 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786008495763054-20932-0/minicluster-data/ts-0/data/instance:
uuid: "0f53e3e20c9c4ecba50ba7034fbd8839"
format_stamp: "Formatted at 2026-08-06 09:33:48 on dist-test-slave-45dx"
server_key: "9acf04d3cacfb67f5703e3b395976940"
server_key_iv: "d1249c134dbc07181ea4a0b5d177fa26"
server_key_version: "encryptionkey@0"
I20260806 09:33:48.596704 17490 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786008495763054-20932-0/minicluster-data/ts-0/wal/instance:
uuid: "0f53e3e20c9c4ecba50ba7034fbd8839"
format_stamp: "Formatted at 2026-08-06 09:33:48 on dist-test-slave-45dx"
server_key: "9acf04d3cacfb67f5703e3b395976940"
server_key_iv: "d1249c134dbc07181ea4a0b5d177fa26"
server_key_version: "encryptionkey@0"
I20260806 09:33:48.601650 17490 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260806 09:33:48.605175 17512 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:33:48.606468 17490 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 09:33:48.606647 17490 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786008495763054-20932-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786008495763054-20932-0/minicluster-data/ts-0/wal
uuid: "0f53e3e20c9c4ecba50ba7034fbd8839"
format_stamp: "Formatted at 2026-08-06 09:33:48 on dist-test-slave-45dx"
server_key: "9acf04d3cacfb67f5703e3b395976940"
server_key_iv: "d1249c134dbc07181ea4a0b5d177fa26"
server_key_version: "encryptionkey@0"
I20260806 09:33:48.606877 17490 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786008495763054-20932-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786008495763054-20932-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786008495763054-20932-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:33:48.618499 17490 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:33:48.619279 17490 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:33:48.619598 17490 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:33:48.620561 17490 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:33:48.621985 17490 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:33:48.622112 17490 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:33:48.622244 17490 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:33:48.622341 17490 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:33:48.657575 17490 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.1:45017
I20260806 09:33:48.657629 17674 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.1:45017 every 8 connection(s)
I20260806 09:33:48.658908 17490 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786008495763054-20932-0/minicluster-data/ts-0/data/info.pb
I20260806 09:33:48.663909 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 17490
I20260806 09:33:48.664096 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786008495763054-20932-0/minicluster-data/ts-0/wal/instance
I20260806 09:33:48.664620 20932 external_mini_cluster.cc:1428] Setting key b0e52ef9e0e59c557d29c999bfbd436a
I20260806 09:33:48.667750 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786008495763054-20932-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786008495763054-20932-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786008495763054-20932-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786008495763054-20932-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.113.2:0
--local_ip_for_outbound_sockets=127.20.113.2
--webserver_interface=127.20.113.2
--webserver_port=0
--tserver_master_addrs=127.20.113.62:42369
--builtin_ntp_servers=127.20.113.20:41597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 09:33:48.672714 17680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:48.661497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.1:49651 (local address 127.20.113.62:42369)
0806 09:33:48.661990 (+ 493us) server_negotiation.cc:206] Beginning negotiation
0806 09:33:48.661996 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:33:48.663177 (+ 1181us) server_negotiation.cc:407] Connection header received
0806 09:33:48.665128 (+ 1951us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:33:48.665136 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:33:48.665227 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:33:48.665329 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:33:48.666711 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:48.667717 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:48.669572 (+ 1855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:48.670336 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:48.670473 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:48.671764 (+ 1291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:33:48.671772 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:33:48.671780 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:33:48.671805 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:33:48.671861 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:33:48.671880 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:33:48.671974 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 09:33:48.672168 (+ 194us) server_negotiation.cc:299] Negotiation successful
0806 09:33:48.672274 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":245,"threads_started":1}
I20260806 09:33:48.673473 17682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:48.662134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:42369 (local address 127.20.113.1:49651)
0806 09:33:48.663007 (+ 873us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:48.663046 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 09:33:48.664847 (+ 1801us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:33:48.665581 (+ 734us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:33:48.665589 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:33:48.665702 (+ 113us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:33:48.666494 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:33:48.666519 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:33:48.667952 (+ 1433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:33:48.667960 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:33:48.669396 (+ 1436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:33:48.669409 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:33:48.670528 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:33:48.670532 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:33:48.670771 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:48.671461 (+ 690us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 09:33:48.671464 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:33:48.671499 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 09:33:48.671566 (+ 67us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:33:48.671978 (+ 412us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:33:48.671987 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:33:48.671992 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 09:33:48.672118 (+ 126us) client_negotiation.cc:240] Negotiation successful
0806 09:33:48.672448 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":287,"threads_started":1}
I20260806 09:33:48.674930 17676 heartbeater.cc:347] Connected to a master server at 127.20.113.62:42369
I20260806 09:33:48.675225 17676 heartbeater.cc:464] Registering TS with master...
I20260806 09:33:48.675904 17676 heartbeater.cc:511] Master 127.20.113.62:42369 requested a full tablet report, sending...
I20260806 09:33:48.677884 17405 ts_manager.cc:194] Registered new tserver with Master: 0f53e3e20c9c4ecba50ba7034fbd8839 (127.20.113.1:45017)
I20260806 09:33:48.679426 17405 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.113.1:49651
W20260806 09:33:49.021682 17683 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:33:49.021978 17683 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:33:49.022065 17683 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:33:49.028833 17683 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:33:49.028921 17683 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:33:49.029021 17683 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.113.2
I20260806 09:33:49.037591 17683 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:41597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786008495763054-20932-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786008495763054-20932-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.113.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786008495763054-20932-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.113.2
--webserver_port=0
--tserver_master_addrs=127.20.113.62:42369
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.113.2
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786008495763054-20932-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:33:49.038748 17683 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:33:49.040652 17683 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:33:49.054167 17689 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:33:49.054193 17688 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:33:49.054175 17693 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:33:49.055259 17683 server_base.cc:1034] running on GCE node
I20260806 09:33:49.055888 17683 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:33:49.056885 17683 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:33:49.058125 17683 hybrid_clock.cc:648] HybridClock initialized: now 1786008829058082 us; error 48 us; skew 500 ppm
I20260806 09:33:49.058452 17683 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 09:33:49.060724 17683 webserver.cc:460] Webserver started at http://127.20.113.2:38407/ using document root <none> and password file <none>
I20260806 09:33:49.061425 17683 fs_manager.cc:359] Metadata directory not provided
I20260806 09:33:49.061563 17683 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:33:49.061901 17683 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:33:49.064766 17683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786008495763054-20932-0/minicluster-data/ts-1/data/instance:
uuid: "126f98660a9849dcba430c50369debd8"
format_stamp: "Formatted at 2026-08-06 09:33:49 on dist-test-slave-45dx"
server_key: "76a3004df031b5c9e326384f222c4be3"
server_key_iv: "943018c2f92c7903d38167043e315588"
server_key_version: "encryptionkey@0"
I20260806 09:33:49.065623 17683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786008495763054-20932-0/minicluster-data/ts-1/wal/instance:
uuid: "126f98660a9849dcba430c50369debd8"
format_stamp: "Formatted at 2026-08-06 09:33:49 on dist-test-slave-45dx"
server_key: "76a3004df031b5c9e326384f222c4be3"
server_key_iv: "943018c2f92c7903d38167043e315588"
server_key_version: "encryptionkey@0"
I20260806 09:33:49.070745 17683 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260806 09:33:49.073999 17702 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:33:49.075249 17683 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 09:33:49.075461 17683 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786008495763054-20932-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786008495763054-20932-0/minicluster-data/ts-1/wal
uuid: "126f98660a9849dcba430c50369debd8"
format_stamp: "Formatted at 2026-08-06 09:33:49 on dist-test-slave-45dx"
server_key: "76a3004df031b5c9e326384f222c4be3"
server_key_iv: "943018c2f92c7903d38167043e315588"
server_key_version: "encryptionkey@0"
I20260806 09:33:49.075682 17683 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786008495763054-20932-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786008495763054-20932-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786008495763054-20932-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:33:49.097990 17683 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:33:49.098850 17683 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:33:49.099215 17683 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:33:49.100226 17683 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:33:49.101920 17683 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:33:49.102053 17683 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:33:49.102180 17683 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:33:49.102276 17683 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:33:49.137485 17683 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.2:39097
I20260806 09:33:49.137574 17868 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.2:39097 every 8 connection(s)
I20260806 09:33:49.138799 17683 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786008495763054-20932-0/minicluster-data/ts-1/data/info.pb
I20260806 09:33:49.146598 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 17683
I20260806 09:33:49.146777 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786008495763054-20932-0/minicluster-data/ts-1/wal/instance
I20260806 09:33:49.147300 20932 external_mini_cluster.cc:1428] Setting key 5c892a67da1b9fe3c90c1265080661c9
I20260806 09:33:49.150239 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786008495763054-20932-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786008495763054-20932-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786008495763054-20932-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786008495763054-20932-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.113.3:0
--local_ip_for_outbound_sockets=127.20.113.3
--webserver_interface=127.20.113.3
--webserver_port=0
--tserver_master_addrs=127.20.113.62:42369
--builtin_ntp_servers=127.20.113.20:41597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 09:33:49.151993 17680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:49.141483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.2:46041 (local address 127.20.113.62:42369)
0806 09:33:49.141697 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0806 09:33:49.141704 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:33:49.143160 (+ 1456us) server_negotiation.cc:407] Connection header received
0806 09:33:49.145046 (+ 1886us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:33:49.145051 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:33:49.145120 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:33:49.145208 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:33:49.146418 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:49.147483 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:49.149219 (+ 1736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:49.149605 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:49.149741 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:49.151143 (+ 1402us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:33:49.151148 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:33:49.151152 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:33:49.151169 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:33:49.151211 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:33:49.151222 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:33:49.151323 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 09:33:49.151552 (+ 229us) server_negotiation.cc:299] Negotiation successful
0806 09:33:49.151661 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 09:33:49.153038 17871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:49.142113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:42369 (local address 127.20.113.2:46041)
0806 09:33:49.142999 (+ 886us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:49.143034 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 09:33:49.144805 (+ 1771us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:33:49.145465 (+ 660us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:33:49.145473 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:33:49.145592 (+ 119us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:33:49.146235 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:33:49.146255 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:33:49.147620 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:33:49.147624 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:33:49.149074 (+ 1450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:33:49.149090 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:33:49.149835 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:33:49.149843 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:33:49.150066 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:49.150838 (+ 772us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 09:33:49.150842 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:33:49.150881 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 09:33:49.150958 (+ 77us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:33:49.151324 (+ 366us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:33:49.151350 (+ 26us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:33:49.151363 (+ 13us) client_negotiation.cc:769] Sending connection context
0806 09:33:49.151539 (+ 176us) client_negotiation.cc:240] Negotiation successful
0806 09:33:49.151937 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":295,"threads_started":1}
I20260806 09:33:49.154470 17869 heartbeater.cc:347] Connected to a master server at 127.20.113.62:42369
I20260806 09:33:49.154768 17869 heartbeater.cc:464] Registering TS with master...
I20260806 09:33:49.155484 17869 heartbeater.cc:511] Master 127.20.113.62:42369 requested a full tablet report, sending...
I20260806 09:33:49.156919 17405 ts_manager.cc:194] Registered new tserver with Master: 126f98660a9849dcba430c50369debd8 (127.20.113.2:39097)
I20260806 09:33:49.158104 17405 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.113.2:46041
W20260806 09:33:49.507184 17872 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:33:49.507596 17872 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:33:49.507781 17872 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:33:49.516968 17872 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:33:49.517072 17872 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:33:49.517197 17872 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.113.3
I20260806 09:33:49.526997 17872 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:41597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786008495763054-20932-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786008495763054-20932-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.113.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786008495763054-20932-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.113.3
--webserver_port=0
--tserver_master_addrs=127.20.113.62:42369
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.113.3
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786008495763054-20932-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:33:49.528236 17872 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:33:49.530170 17872 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:33:49.543196 17882 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:33:49.543220 17884 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:33:49.543293 17887 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:33:49.544240 17872 server_base.cc:1034] running on GCE node
I20260806 09:33:49.544925 17872 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:33:49.546121 17872 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:33:49.547379 17872 hybrid_clock.cc:648] HybridClock initialized: now 1786008829547334 us; error 25 us; skew 500 ppm
I20260806 09:33:49.547745 17872 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 09:33:49.550045 17872 webserver.cc:460] Webserver started at http://127.20.113.3:43489/ using document root <none> and password file <none>
I20260806 09:33:49.550734 17872 fs_manager.cc:359] Metadata directory not provided
I20260806 09:33:49.550877 17872 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:33:49.551198 17872 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:33:49.554173 17872 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786008495763054-20932-0/minicluster-data/ts-2/data/instance:
uuid: "b51d72485b634e3795173bd1c98077bc"
format_stamp: "Formatted at 2026-08-06 09:33:49 on dist-test-slave-45dx"
server_key: "e1c11bfbf7c92f8c2807919b54707aa1"
server_key_iv: "ce034794cec327c9862255bbd5223938"
server_key_version: "encryptionkey@0"
I20260806 09:33:49.554961 17872 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786008495763054-20932-0/minicluster-data/ts-2/wal/instance:
uuid: "b51d72485b634e3795173bd1c98077bc"
format_stamp: "Formatted at 2026-08-06 09:33:49 on dist-test-slave-45dx"
server_key: "e1c11bfbf7c92f8c2807919b54707aa1"
server_key_iv: "ce034794cec327c9862255bbd5223938"
server_key_version: "encryptionkey@0"
I20260806 09:33:49.559955 17872 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260806 09:33:49.563297 17894 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:33:49.564576 17872 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 09:33:49.564746 17872 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786008495763054-20932-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786008495763054-20932-0/minicluster-data/ts-2/wal
uuid: "b51d72485b634e3795173bd1c98077bc"
format_stamp: "Formatted at 2026-08-06 09:33:49 on dist-test-slave-45dx"
server_key: "e1c11bfbf7c92f8c2807919b54707aa1"
server_key_iv: "ce034794cec327c9862255bbd5223938"
server_key_version: "encryptionkey@0"
I20260806 09:33:49.564956 17872 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786008495763054-20932-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786008495763054-20932-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786008495763054-20932-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:33:49.582015 17872 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:33:49.582743 17872 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:33:49.583057 17872 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:33:49.583976 17872 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 09:33:49.585436 17872 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 09:33:49.585575 17872 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 09:33:49.585696 17872 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 09:33:49.585785 17872 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 09:33:49.619855 17872 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.3:43555
I20260806 09:33:49.619911 18066 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.3:43555 every 8 connection(s)
I20260806 09:33:49.621273 17872 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786008495763054-20932-0/minicluster-data/ts-2/data/info.pb
I20260806 09:33:49.627722 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 17872
I20260806 09:33:49.627935 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786008495763054-20932-0/minicluster-data/ts-2/wal/instance
I20260806 09:33:49.628490 20932 external_mini_cluster.cc:1428] Setting key cbeb31d1dde305a6022dbbb17e5a508b
I20260806 09:33:49.634563 17680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:49.624002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.3:38319 (local address 127.20.113.62:42369)
0806 09:33:49.624182 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0806 09:33:49.624186 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 09:33:49.625659 (+ 1473us) server_negotiation.cc:407] Connection header received
0806 09:33:49.627568 (+ 1909us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:33:49.627574 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:33:49.627675 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:33:49.627784 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:33:49.629211 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:49.630295 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:49.632178 (+ 1883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:49.632640 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:49.632740 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:49.633922 (+ 1182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:33:49.633927 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:33:49.633930 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:33:49.633946 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:33:49.633977 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:33:49.633988 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:33:49.634055 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 09:33:49.634241 (+ 186us) server_negotiation.cc:299] Negotiation successful
0806 09:33:49.634313 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 09:33:49.635450 18071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:49.624586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:42369 (local address 127.20.113.3:38319)
0806 09:33:49.625478 (+ 892us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:49.625540 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0806 09:33:49.627230 (+ 1690us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:33:49.628076 (+ 846us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:33:49.628088 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:33:49.628183 (+ 95us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:33:49.629003 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:33:49.629031 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:33:49.630480 (+ 1449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:33:49.630489 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:33:49.632048 (+ 1559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:33:49.632060 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:33:49.632826 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:33:49.632832 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:33:49.633014 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:49.633634 (+ 620us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 09:33:49.633637 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:33:49.633673 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 09:33:49.633741 (+ 68us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:33:49.634070 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:33:49.634078 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:33:49.634084 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 09:33:49.634213 (+ 129us) client_negotiation.cc:240] Negotiation successful
0806 09:33:49.634483 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":287,"threads_started":1}
I20260806 09:33:49.636754 18067 heartbeater.cc:347] Connected to a master server at 127.20.113.62:42369
I20260806 09:33:49.636999 18067 heartbeater.cc:464] Registering TS with master...
I20260806 09:33:49.637724 18067 heartbeater.cc:511] Master 127.20.113.62:42369 requested a full tablet report, sending...
I20260806 09:33:49.639038 17405 ts_manager.cc:194] Registered new tserver with Master: b51d72485b634e3795173bd1c98077bc (127.20.113.3:43555)
I20260806 09:33:49.639837 17405 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.113.3:38319
I20260806 09:33:49.643404 20932 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 09:33:49.652977 17680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:49.647470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44296 (local address 127.20.113.62:42369)
0806 09:33:49.647661 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0806 09:33:49.647668 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:33:49.648124 (+ 456us) server_negotiation.cc:407] Connection header received
0806 09:33:49.648245 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:33:49.648249 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:33:49.648377 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:33:49.648464 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:33:49.648904 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:49.649608 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:49.650841 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:49.651230 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:49.651314 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:49.652352 (+ 1038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:33:49.652364 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:33:49.652367 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:33:49.652383 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:33:49.652411 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:33:49.652422 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:33:49.652485 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 09:33:49.652611 (+ 126us) server_negotiation.cc:299] Negotiation successful
0806 09:33:49.652678 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 09:33:49.656724 17405 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44296:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 09:33:49.663482 17405 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 09:33:49.682116 17676 heartbeater.cc:503] Master 127.20.113.62:42369 was elected leader, sending a full tablet report...
I20260806 09:33:49.685177 18087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:49.674455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.3:43555 (local address 127.0.0.1:52820)
0806 09:33:49.675161 (+ 706us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:49.675200 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 09:33:49.675365 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:33:49.677231 (+ 1866us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:33:49.677238 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:33:49.677273 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:33:49.677697 (+ 424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:33:49.677714 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:33:49.680010 (+ 2296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:33:49.680015 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:33:49.682383 (+ 2368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:33:49.682397 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:33:49.684237 (+ 1840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:33:49.684244 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:33:49.684500 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:49.684549 (+ 49us) client_negotiation.cc:769] Sending connection context
0806 09:33:49.684616 (+ 67us) client_negotiation.cc:240] Negotiation successful
0806 09:33:49.684802 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":477,"mutex_wait_us":64,"thread_start_us":305,"threads_started":1}
I20260806 09:33:49.685165 18093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:49.676343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32986 (local address 127.20.113.1:45017)
0806 09:33:49.677485 (+ 1142us) server_negotiation.cc:206] Beginning negotiation
0806 09:33:49.677494 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 09:33:49.677525 (+ 31us) server_negotiation.cc:407] Connection header received
0806 09:33:49.677628 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:33:49.677637 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:33:49.677846 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:33:49.678745 (+ 899us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:33:49.678966 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:49.680350 (+ 1384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:49.682728 (+ 2378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:49.683896 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:49.684070 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:49.684384 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0806 09:33:49.684639 (+ 255us) server_negotiation.cc:299] Negotiation successful
0806 09:33:49.684803 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":921,"thread_start_us":786,"threads_started":1}
I20260806 09:33:49.685585 18088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:49.674673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52820 (local address 127.20.113.3:43555)
0806 09:33:49.676562 (+ 1889us) server_negotiation.cc:206] Beginning negotiation
0806 09:33:49.676572 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 09:33:49.676613 (+ 41us) server_negotiation.cc:407] Connection header received
0806 09:33:49.676724 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:33:49.676733 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:33:49.676987 (+ 254us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:33:49.677928 (+ 941us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:33:49.678233 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:49.679841 (+ 1608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:49.682598 (+ 2757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:49.683997 (+ 1399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:49.684726 (+ 729us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:49.684994 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0806 09:33:49.685067 (+ 73us) server_negotiation.cc:299] Negotiation successful
0806 09:33:49.685218 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1724,"thread_start_us":1563,"threads_started":1}
I20260806 09:33:49.686141 18089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:49.675549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.1:45017 (local address 127.0.0.1:32986)
0806 09:33:49.676862 (+ 1313us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:49.676879 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 09:33:49.676984 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:33:49.678064 (+ 1080us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:33:49.678068 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:33:49.678091 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:33:49.678552 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:33:49.678565 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:33:49.680559 (+ 1994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:33:49.680566 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:33:49.682499 (+ 1933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:33:49.682515 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:33:49.684100 (+ 1585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:33:49.684106 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:33:49.684368 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:49.684425 (+ 57us) client_negotiation.cc:769] Sending connection context
0806 09:33:49.684597 (+ 172us) client_negotiation.cc:240] Negotiation successful
0806 09:33:49.685898 (+ 1301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1204,"mutex_wait_us":23,"thread_start_us":326,"threads_started":1}
I20260806 09:33:49.687168 18090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:49.675099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.2:39097 (local address 127.0.0.1:34982)
0806 09:33:49.676182 (+ 1083us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:49.676202 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 09:33:49.676346 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:33:49.677469 (+ 1123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:33:49.677473 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:33:49.677504 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:33:49.677942 (+ 438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:33:49.677960 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:33:49.679740 (+ 1780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:33:49.679749 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:33:49.683596 (+ 3847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:33:49.683609 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:33:49.686472 (+ 2863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:33:49.686484 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:33:49.686650 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:49.686682 (+ 32us) client_negotiation.cc:769] Sending connection context
0806 09:33:49.686757 (+ 75us) client_negotiation.cc:240] Negotiation successful
0806 09:33:49.686875 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":922,"mutex_wait_us":80,"thread_start_us":1666,"threads_started":1}
I20260806 09:33:49.688392 18091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:49.675712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34982 (local address 127.20.113.2:39097)
0806 09:33:49.676869 (+ 1157us) server_negotiation.cc:206] Beginning negotiation
0806 09:33:49.676877 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 09:33:49.676908 (+ 31us) server_negotiation.cc:407] Connection header received
0806 09:33:49.677035 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:33:49.677046 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:33:49.677286 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:33:49.677480 (+ 194us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:33:49.678324 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:49.679622 (+ 1298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:49.683815 (+ 4193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:49.685275 (+ 1460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:49.687530 (+ 2255us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:49.687777 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0806 09:33:49.687844 (+ 67us) server_negotiation.cc:299] Negotiation successful
0806 09:33:49.688012 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":997,"thread_start_us":865,"threads_started":1}
I20260806 09:33:49.689443 17581 tablet_service.cc:1467] Processing CreateTablet for tablet cbd4bc32e4c342c1bb978f62bec5ee08 (DEFAULT_TABLE table=test-table [id=0acd2b20aa8148739a17e90c337b7f5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:33:49.689383 17967 tablet_service.cc:1467] Processing CreateTablet for tablet cbd4bc32e4c342c1bb978f62bec5ee08 (DEFAULT_TABLE table=test-table [id=0acd2b20aa8148739a17e90c337b7f5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:33:49.690753 17581 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cbd4bc32e4c342c1bb978f62bec5ee08. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:33:49.690798 17967 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cbd4bc32e4c342c1bb978f62bec5ee08. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:33:49.691895 17768 tablet_service.cc:1467] Processing CreateTablet for tablet cbd4bc32e4c342c1bb978f62bec5ee08 (DEFAULT_TABLE table=test-table [id=0acd2b20aa8148739a17e90c337b7f5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 09:33:49.693571 17768 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cbd4bc32e4c342c1bb978f62bec5ee08. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:33:49.701692 18095 tablet_bootstrap.cc:492] T cbd4bc32e4c342c1bb978f62bec5ee08 P 0f53e3e20c9c4ecba50ba7034fbd8839: Bootstrap starting.
I20260806 09:33:49.705241 18096 tablet_bootstrap.cc:492] T cbd4bc32e4c342c1bb978f62bec5ee08 P b51d72485b634e3795173bd1c98077bc: Bootstrap starting.
I20260806 09:33:49.706458 18095 tablet_bootstrap.cc:654] T cbd4bc32e4c342c1bb978f62bec5ee08 P 0f53e3e20c9c4ecba50ba7034fbd8839: Neither blocks nor log segments found. Creating new log.
I20260806 09:33:49.707846 18095 log.cc:824] T cbd4bc32e4c342c1bb978f62bec5ee08 P 0f53e3e20c9c4ecba50ba7034fbd8839: Log is configured to *not* fsync() on all Append() calls
I20260806 09:33:49.708122 18097 tablet_bootstrap.cc:492] T cbd4bc32e4c342c1bb978f62bec5ee08 P 126f98660a9849dcba430c50369debd8: Bootstrap starting.
I20260806 09:33:49.709122 18096 tablet_bootstrap.cc:654] T cbd4bc32e4c342c1bb978f62bec5ee08 P b51d72485b634e3795173bd1c98077bc: Neither blocks nor log segments found. Creating new log.
I20260806 09:33:49.710623 18096 log.cc:824] T cbd4bc32e4c342c1bb978f62bec5ee08 P b51d72485b634e3795173bd1c98077bc: Log is configured to *not* fsync() on all Append() calls
I20260806 09:33:49.711364 18097 tablet_bootstrap.cc:654] T cbd4bc32e4c342c1bb978f62bec5ee08 P 126f98660a9849dcba430c50369debd8: Neither blocks nor log segments found. Creating new log.
I20260806 09:33:49.712754 18097 log.cc:824] T cbd4bc32e4c342c1bb978f62bec5ee08 P 126f98660a9849dcba430c50369debd8: Log is configured to *not* fsync() on all Append() calls
I20260806 09:33:49.714252 18096 tablet_bootstrap.cc:492] T cbd4bc32e4c342c1bb978f62bec5ee08 P b51d72485b634e3795173bd1c98077bc: No bootstrap required, opened a new log
I20260806 09:33:49.714386 18095 tablet_bootstrap.cc:492] T cbd4bc32e4c342c1bb978f62bec5ee08 P 0f53e3e20c9c4ecba50ba7034fbd8839: No bootstrap required, opened a new log
I20260806 09:33:49.714622 18096 ts_tablet_manager.cc:1397] T cbd4bc32e4c342c1bb978f62bec5ee08 P b51d72485b634e3795173bd1c98077bc: Time spent bootstrapping tablet: real 0.010s user 0.009s sys 0.000s
I20260806 09:33:49.714766 18095 ts_tablet_manager.cc:1397] T cbd4bc32e4c342c1bb978f62bec5ee08 P 0f53e3e20c9c4ecba50ba7034fbd8839: Time spent bootstrapping tablet: real 0.013s user 0.011s sys 0.000s
I20260806 09:33:49.715803 18097 tablet_bootstrap.cc:492] T cbd4bc32e4c342c1bb978f62bec5ee08 P 126f98660a9849dcba430c50369debd8: No bootstrap required, opened a new log
I20260806 09:33:49.716058 18097 ts_tablet_manager.cc:1397] T cbd4bc32e4c342c1bb978f62bec5ee08 P 126f98660a9849dcba430c50369debd8: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260806 09:33:49.721807 18096 raft_consensus.cc:348] T cbd4bc32e4c342c1bb978f62bec5ee08 P b51d72485b634e3795173bd1c98077bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b51d72485b634e3795173bd1c98077bc" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 43555 } } peers { permanent_uuid: "0f53e3e20c9c4ecba50ba7034fbd8839" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 45017 } } peers { permanent_uuid: "126f98660a9849dcba430c50369debd8" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 39097 } }
I20260806 09:33:49.722208 18096 raft_consensus.cc:374] T cbd4bc32e4c342c1bb978f62bec5ee08 P b51d72485b634e3795173bd1c98077bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:33:49.722339 18096 raft_consensus.cc:729] T cbd4bc32e4c342c1bb978f62bec5ee08 P b51d72485b634e3795173bd1c98077bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b51d72485b634e3795173bd1c98077bc, State: Initialized, Role: FOLLOWER
I20260806 09:33:49.722967 18096 consensus_queue.cc:260] T cbd4bc32e4c342c1bb978f62bec5ee08 P b51d72485b634e3795173bd1c98077bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b51d72485b634e3795173bd1c98077bc" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 43555 } } peers { permanent_uuid: "0f53e3e20c9c4ecba50ba7034fbd8839" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 45017 } } peers { permanent_uuid: "126f98660a9849dcba430c50369debd8" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 39097 } }
I20260806 09:33:49.724097 18067 heartbeater.cc:503] Master 127.20.113.62:42369 was elected leader, sending a full tablet report...
I20260806 09:33:49.724793 18096 ts_tablet_manager.cc:1428] T cbd4bc32e4c342c1bb978f62bec5ee08 P b51d72485b634e3795173bd1c98077bc: Time spent starting tablet: real 0.010s user 0.003s sys 0.006s
I20260806 09:33:49.725373 18095 raft_consensus.cc:348] T cbd4bc32e4c342c1bb978f62bec5ee08 P 0f53e3e20c9c4ecba50ba7034fbd8839 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b51d72485b634e3795173bd1c98077bc" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 43555 } } peers { permanent_uuid: "0f53e3e20c9c4ecba50ba7034fbd8839" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 45017 } } peers { permanent_uuid: "126f98660a9849dcba430c50369debd8" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 39097 } }
I20260806 09:33:49.725849 18095 raft_consensus.cc:374] T cbd4bc32e4c342c1bb978f62bec5ee08 P 0f53e3e20c9c4ecba50ba7034fbd8839 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:33:49.725992 18095 raft_consensus.cc:729] T cbd4bc32e4c342c1bb978f62bec5ee08 P 0f53e3e20c9c4ecba50ba7034fbd8839 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f53e3e20c9c4ecba50ba7034fbd8839, State: Initialized, Role: FOLLOWER
I20260806 09:33:49.726465 18097 raft_consensus.cc:348] T cbd4bc32e4c342c1bb978f62bec5ee08 P 126f98660a9849dcba430c50369debd8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b51d72485b634e3795173bd1c98077bc" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 43555 } } peers { permanent_uuid: "0f53e3e20c9c4ecba50ba7034fbd8839" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 45017 } } peers { permanent_uuid: "126f98660a9849dcba430c50369debd8" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 39097 } }
I20260806 09:33:49.726877 18097 raft_consensus.cc:374] T cbd4bc32e4c342c1bb978f62bec5ee08 P 126f98660a9849dcba430c50369debd8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:33:49.726729 18095 consensus_queue.cc:260] T cbd4bc32e4c342c1bb978f62bec5ee08 P 0f53e3e20c9c4ecba50ba7034fbd8839 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b51d72485b634e3795173bd1c98077bc" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 43555 } } peers { permanent_uuid: "0f53e3e20c9c4ecba50ba7034fbd8839" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 45017 } } peers { permanent_uuid: "126f98660a9849dcba430c50369debd8" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 39097 } }
I20260806 09:33:49.726977 18097 raft_consensus.cc:729] T cbd4bc32e4c342c1bb978f62bec5ee08 P 126f98660a9849dcba430c50369debd8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 126f98660a9849dcba430c50369debd8, State: Initialized, Role: FOLLOWER
I20260806 09:33:49.727597 18097 consensus_queue.cc:260] T cbd4bc32e4c342c1bb978f62bec5ee08 P 126f98660a9849dcba430c50369debd8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b51d72485b634e3795173bd1c98077bc" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 43555 } } peers { permanent_uuid: "0f53e3e20c9c4ecba50ba7034fbd8839" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 45017 } } peers { permanent_uuid: "126f98660a9849dcba430c50369debd8" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 39097 } }
I20260806 09:33:49.728811 18095 ts_tablet_manager.cc:1428] T cbd4bc32e4c342c1bb978f62bec5ee08 P 0f53e3e20c9c4ecba50ba7034fbd8839: Time spent starting tablet: real 0.014s user 0.008s sys 0.005s
I20260806 09:33:49.729198 17869 heartbeater.cc:503] Master 127.20.113.62:42369 was elected leader, sending a full tablet report...
I20260806 09:33:49.729799 18097 ts_tablet_manager.cc:1428] T cbd4bc32e4c342c1bb978f62bec5ee08 P 126f98660a9849dcba430c50369debd8: Time spent starting tablet: real 0.014s user 0.007s sys 0.008s
W20260806 09:33:49.873235 18069 tablet.cc:2369] T cbd4bc32e4c342c1bb978f62bec5ee08 P b51d72485b634e3795173bd1c98077bc: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 09:33:49.891062 17870 tablet.cc:2369] T cbd4bc32e4c342c1bb978f62bec5ee08 P 126f98660a9849dcba430c50369debd8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 09:33:49.911568 17677 tablet.cc:2369] T cbd4bc32e4c342c1bb978f62bec5ee08 P 0f53e3e20c9c4ecba50ba7034fbd8839: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 09:33:49.956847 18106 raft_consensus.cc:482] T cbd4bc32e4c342c1bb978f62bec5ee08 P b51d72485b634e3795173bd1c98077bc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 09:33:49.957170 18106 raft_consensus.cc:504] T cbd4bc32e4c342c1bb978f62bec5ee08 P b51d72485b634e3795173bd1c98077bc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b51d72485b634e3795173bd1c98077bc" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 43555 } } peers { permanent_uuid: "0f53e3e20c9c4ecba50ba7034fbd8839" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 45017 } } peers { permanent_uuid: "126f98660a9849dcba430c50369debd8" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 39097 } }
I20260806 09:33:49.958953 18106 leader_election.cc:288] T cbd4bc32e4c342c1bb978f62bec5ee08 P b51d72485b634e3795173bd1c98077bc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0f53e3e20c9c4ecba50ba7034fbd8839 (127.20.113.1:45017), 126f98660a9849dcba430c50369debd8 (127.20.113.2:39097)
I20260806 09:33:49.965984 18071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:49.959409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.1:45017 (local address 127.20.113.3:41369)
0806 09:33:49.959741 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:49.959780 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 09:33:49.959935 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:33:49.960379 (+ 444us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:33:49.960384 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:33:49.960415 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:33:49.960659 (+ 244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:33:49.960669 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:33:49.961932 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:33:49.961935 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:33:49.963992 (+ 2057us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:33:49.964008 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:33:49.965368 (+ 1360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:33:49.965372 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:33:49.965488 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:49.965512 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 09:33:49.965586 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 09:33:49.965677 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"mutex_wait_us":4}
I20260806 09:33:49.966030 18093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:49.959351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.3:41369 (local address 127.20.113.1:45017)
0806 09:33:49.959588 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0806 09:33:49.959596 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 09:33:49.960057 (+ 461us) server_negotiation.cc:407] Connection header received
0806 09:33:49.960124 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:33:49.960127 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:33:49.960195 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:33:49.960260 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:33:49.960849 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:49.961780 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:49.964140 (+ 2360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:49.965209 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:49.965367 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:49.965441 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 09:33:49.965638 (+ 197us) server_negotiation.cc:299] Negotiation successful
0806 09:33:49.965770 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260806 09:33:49.966621 18091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:49.959708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.113.3:42009 (local address 127.20.113.2:39097)
0806 09:33:49.959922 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0806 09:33:49.959928 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:33:49.960404 (+ 476us) server_negotiation.cc:407] Connection header received
0806 09:33:49.960670 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:33:49.960674 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:33:49.960785 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:33:49.960881 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 09:33:49.961396 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:49.962281 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:49.964625 (+ 2344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:49.965762 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:49.965917 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:49.965976 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 09:33:49.966315 (+ 339us) server_negotiation.cc:299] Negotiation successful
0806 09:33:49.966427 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260806 09:33:49.967398 18112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:49.959491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.2:39097 (local address 127.20.113.3:42009)
0806 09:33:49.960275 (+ 784us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:49.960294 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 09:33:49.960518 (+ 224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:33:49.960907 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:33:49.960910 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:33:49.960932 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 09:33:49.961221 (+ 289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:33:49.961227 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:33:49.962428 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:33:49.962433 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:33:49.964443 (+ 2010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:33:49.964455 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:33:49.965933 (+ 1478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:33:49.965939 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:33:49.966125 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:49.966168 (+ 43us) client_negotiation.cc:769] Sending connection context
0806 09:33:49.966869 (+ 701us) client_negotiation.cc:240] Negotiation successful
0806 09:33:49.967029 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":642,"spinlock_wait_cycles":34560,"thread_start_us":456,"threads_started":1}
I20260806 09:33:49.967244 17611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cbd4bc32e4c342c1bb978f62bec5ee08" candidate_uuid: "b51d72485b634e3795173bd1c98077bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f53e3e20c9c4ecba50ba7034fbd8839" is_pre_election: true
I20260806 09:33:49.967905 17611 raft_consensus.cc:2454] T cbd4bc32e4c342c1bb978f62bec5ee08 P 0f53e3e20c9c4ecba50ba7034fbd8839 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b51d72485b634e3795173bd1c98077bc in term 0.
I20260806 09:33:49.968782 17897 leader_election.cc:302] T cbd4bc32e4c342c1bb978f62bec5ee08 P b51d72485b634e3795173bd1c98077bc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f53e3e20c9c4ecba50ba7034fbd8839, b51d72485b634e3795173bd1c98077bc; no voters:
I20260806 09:33:49.968664 17795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cbd4bc32e4c342c1bb978f62bec5ee08" candidate_uuid: "b51d72485b634e3795173bd1c98077bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "126f98660a9849dcba430c50369debd8" is_pre_election: true
I20260806 09:33:49.969287 18106 raft_consensus.cc:2781] T cbd4bc32e4c342c1bb978f62bec5ee08 P b51d72485b634e3795173bd1c98077bc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 09:33:49.969244 17795 raft_consensus.cc:2454] T cbd4bc32e4c342c1bb978f62bec5ee08 P 126f98660a9849dcba430c50369debd8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b51d72485b634e3795173bd1c98077bc in term 0.
I20260806 09:33:49.969460 18106 raft_consensus.cc:482] T cbd4bc32e4c342c1bb978f62bec5ee08 P b51d72485b634e3795173bd1c98077bc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 09:33:49.969583 18106 raft_consensus.cc:3037] T cbd4bc32e4c342c1bb978f62bec5ee08 P b51d72485b634e3795173bd1c98077bc [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:33:49.973244 18106 raft_consensus.cc:504] T cbd4bc32e4c342c1bb978f62bec5ee08 P b51d72485b634e3795173bd1c98077bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b51d72485b634e3795173bd1c98077bc" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 43555 } } peers { permanent_uuid: "0f53e3e20c9c4ecba50ba7034fbd8839" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 45017 } } peers { permanent_uuid: "126f98660a9849dcba430c50369debd8" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 39097 } }
I20260806 09:33:49.974036 18106 leader_election.cc:288] T cbd4bc32e4c342c1bb978f62bec5ee08 P b51d72485b634e3795173bd1c98077bc [CANDIDATE]: Term 1 election: Requested vote from peers 0f53e3e20c9c4ecba50ba7034fbd8839 (127.20.113.1:45017), 126f98660a9849dcba430c50369debd8 (127.20.113.2:39097)
I20260806 09:33:49.974354 17611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cbd4bc32e4c342c1bb978f62bec5ee08" candidate_uuid: "b51d72485b634e3795173bd1c98077bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f53e3e20c9c4ecba50ba7034fbd8839"
I20260806 09:33:49.974561 17611 raft_consensus.cc:3037] T cbd4bc32e4c342c1bb978f62bec5ee08 P 0f53e3e20c9c4ecba50ba7034fbd8839 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:33:49.974540 17795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cbd4bc32e4c342c1bb978f62bec5ee08" candidate_uuid: "b51d72485b634e3795173bd1c98077bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "126f98660a9849dcba430c50369debd8"
I20260806 09:33:49.974733 17795 raft_consensus.cc:3037] T cbd4bc32e4c342c1bb978f62bec5ee08 P 126f98660a9849dcba430c50369debd8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:33:49.977438 17611 raft_consensus.cc:2454] T cbd4bc32e4c342c1bb978f62bec5ee08 P 0f53e3e20c9c4ecba50ba7034fbd8839 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b51d72485b634e3795173bd1c98077bc in term 1.
I20260806 09:33:49.977560 17795 raft_consensus.cc:2454] T cbd4bc32e4c342c1bb978f62bec5ee08 P 126f98660a9849dcba430c50369debd8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b51d72485b634e3795173bd1c98077bc in term 1.
I20260806 09:33:49.977834 17897 leader_election.cc:302] T cbd4bc32e4c342c1bb978f62bec5ee08 P b51d72485b634e3795173bd1c98077bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f53e3e20c9c4ecba50ba7034fbd8839, b51d72485b634e3795173bd1c98077bc; no voters:
I20260806 09:33:49.978128 18106 raft_consensus.cc:2781] T cbd4bc32e4c342c1bb978f62bec5ee08 P b51d72485b634e3795173bd1c98077bc [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:33:49.978585 18106 raft_consensus.cc:686] T cbd4bc32e4c342c1bb978f62bec5ee08 P b51d72485b634e3795173bd1c98077bc [term 1 LEADER]: Becoming Leader. State: Replica: b51d72485b634e3795173bd1c98077bc, State: Running, Role: LEADER
I20260806 09:33:49.979142 18106 consensus_queue.cc:237] T cbd4bc32e4c342c1bb978f62bec5ee08 P b51d72485b634e3795173bd1c98077bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b51d72485b634e3795173bd1c98077bc" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 43555 } } peers { permanent_uuid: "0f53e3e20c9c4ecba50ba7034fbd8839" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 45017 } } peers { permanent_uuid: "126f98660a9849dcba430c50369debd8" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 39097 } }
I20260806 09:33:49.983629 17405 catalog_manager.cc:5495] T cbd4bc32e4c342c1bb978f62bec5ee08 P b51d72485b634e3795173bd1c98077bc reported cstate change: term changed from 0 to 1, leader changed from <none> to b51d72485b634e3795173bd1c98077bc (127.20.113.3). New cstate: current_term: 1 leader_uuid: "b51d72485b634e3795173bd1c98077bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b51d72485b634e3795173bd1c98077bc" member_type: VOTER last_known_addr { host: "127.20.113.3" port: 43555 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f53e3e20c9c4ecba50ba7034fbd8839" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 45017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "126f98660a9849dcba430c50369debd8" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 39097 } health_report { overall_health: UNKNOWN } } }
I20260806 09:33:50.051875 18088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:50.046442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52828 (local address 127.20.113.3:43555)
0806 09:33:50.046663 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0806 09:33:50.046669 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:33:50.046715 (+ 46us) server_negotiation.cc:407] Connection header received
0806 09:33:50.046952 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:33:50.046956 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:33:50.047046 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:33:50.047172 (+ 126us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 09:33:50.047530 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:50.048241 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:50.049698 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:50.050139 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:50.050297 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:50.050788 (+ 491us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 09:33:50.051283 (+ 495us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 09:33:50.051441 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0806 09:33:50.051539 (+ 98us) server_negotiation.cc:299] Negotiation successful
0806 09:33:50.051619 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 09:33:50.060285 17795 raft_consensus.cc:1261] T cbd4bc32e4c342c1bb978f62bec5ee08 P 126f98660a9849dcba430c50369debd8 [term 1 FOLLOWER]: Refusing update from remote peer b51d72485b634e3795173bd1c98077bc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 09:33:50.060285 17611 raft_consensus.cc:1261] T cbd4bc32e4c342c1bb978f62bec5ee08 P 0f53e3e20c9c4ecba50ba7034fbd8839 [term 1 FOLLOWER]: Refusing update from remote peer b51d72485b634e3795173bd1c98077bc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 09:33:50.061424 18113 consensus_queue.cc:1035] T cbd4bc32e4c342c1bb978f62bec5ee08 P b51d72485b634e3795173bd1c98077bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f53e3e20c9c4ecba50ba7034fbd8839" member_type: VOTER last_known_addr { host: "127.20.113.1" port: 45017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 09:33:50.061861 18106 consensus_queue.cc:1035] T cbd4bc32e4c342c1bb978f62bec5ee08 P b51d72485b634e3795173bd1c98077bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "126f98660a9849dcba430c50369debd8" member_type: VOTER last_known_addr { host: "127.20.113.2" port: 39097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 09:33:50.074409 18120 mvcc.cc:205] Tried to move back new op lower bound from 7315492167915364352 to 7315492167598665728. Current Snapshot: MvccSnapshot[applied={T|T < 7315492167915364352}]
I20260806 09:33:50.076536 18116 mvcc.cc:205] Tried to move back new op lower bound from 7315492167915364352 to 7315492167598665728. Current Snapshot: MvccSnapshot[applied={T|T < 7315492167915364352 or (T in {7315492167915364352})}]
I20260806 09:33:50.087301 17400 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44296:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 09:33:50.087618 17400 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44296:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 09:33:50.091571 17400 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f7046608bce1422e970b014cd0ee9cd4: Sending DeleteTablet for 3 replicas of tablet cbd4bc32e4c342c1bb978f62bec5ee08
I20260806 09:33:50.092689 17967 tablet_service.cc:1514] Processing DeleteTablet for tablet cbd4bc32e4c342c1bb978f62bec5ee08 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:33:50 UTC) from {username='slave'} at 127.0.0.1:52820
I20260806 09:33:50.092869 17581 tablet_service.cc:1514] Processing DeleteTablet for tablet cbd4bc32e4c342c1bb978f62bec5ee08 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:33:50 UTC) from {username='slave'} at 127.0.0.1:32986
I20260806 09:33:50.093078 17768 tablet_service.cc:1514] Processing DeleteTablet for tablet cbd4bc32e4c342c1bb978f62bec5ee08 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 09:33:50 UTC) from {username='slave'} at 127.0.0.1:34982
I20260806 09:33:50.095052 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 17490
I20260806 09:33:50.112470 18129 tablet_replica.cc:330] T cbd4bc32e4c342c1bb978f62bec5ee08 P b51d72485b634e3795173bd1c98077bc: stopping tablet replica
I20260806 09:33:50.112942 18129 raft_consensus.cc:2229] T cbd4bc32e4c342c1bb978f62bec5ee08 P b51d72485b634e3795173bd1c98077bc [term 1 LEADER]: Raft consensus shutting down.
I20260806 09:33:50.113641 18129 raft_consensus.cc:2258] T cbd4bc32e4c342c1bb978f62bec5ee08 P b51d72485b634e3795173bd1c98077bc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 09:33:50.118269 18093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:50.098732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33000 (local address 127.20.113.1:45017)
0806 09:33:50.098977 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0806 09:33:50.098987 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 09:33:50.101774 (+ 2787us) server_negotiation.cc:407] Connection header received
0806 09:33:50.102549 (+ 775us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:33:50.102555 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:33:50.102668 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:33:50.102735 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:33:50.103704 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:50.104798 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:50.108356 (+ 3558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:50.109249 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:50.109384 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:50.116461 (+ 7077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:33:50.116473 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:33:50.116491 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:33:50.117224 (+ 733us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:33:50.117287 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:33:50.117342 (+ 55us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:33:50.117460 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0806 09:33:50.117718 (+ 258us) server_negotiation.cc:299] Negotiation successful
0806 09:33:50.117844 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260806 09:33:50.118670 18129 ts_tablet_manager.cc:1905] T cbd4bc32e4c342c1bb978f62bec5ee08 P b51d72485b634e3795173bd1c98077bc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 09:33:50.119683 17667 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:33:50.125653 18131 tablet_replica.cc:330] T cbd4bc32e4c342c1bb978f62bec5ee08 P 126f98660a9849dcba430c50369debd8: stopping tablet replica
I20260806 09:33:50.126385 18131 raft_consensus.cc:2229] T cbd4bc32e4c342c1bb978f62bec5ee08 P 126f98660a9849dcba430c50369debd8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 09:33:50.127148 18131 raft_consensus.cc:2258] T cbd4bc32e4c342c1bb978f62bec5ee08 P 126f98660a9849dcba430c50369debd8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 09:33:50.132925 18131 ts_tablet_manager.cc:1905] T cbd4bc32e4c342c1bb978f62bec5ee08 P 126f98660a9849dcba430c50369debd8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 09:33:50.135758 18129 ts_tablet_manager.cc:1918] T cbd4bc32e4c342c1bb978f62bec5ee08 P b51d72485b634e3795173bd1c98077bc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 09:33:50.136090 18129 log.cc:1196] T cbd4bc32e4c342c1bb978f62bec5ee08 P b51d72485b634e3795173bd1c98077bc: Deleting WAL directory at /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786008495763054-20932-0/minicluster-data/ts-2/wal/wals/cbd4bc32e4c342c1bb978f62bec5ee08
I20260806 09:33:50.142601 18131 ts_tablet_manager.cc:1918] T cbd4bc32e4c342c1bb978f62bec5ee08 P 126f98660a9849dcba430c50369debd8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 09:33:50.142853 18131 log.cc:1196] T cbd4bc32e4c342c1bb978f62bec5ee08 P 126f98660a9849dcba430c50369debd8: Deleting WAL directory at /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786008495763054-20932-0/minicluster-data/ts-1/wal/wals/cbd4bc32e4c342c1bb978f62bec5ee08
I20260806 09:33:50.143460 18131 ts_tablet_manager.cc:1939] T cbd4bc32e4c342c1bb978f62bec5ee08 P 126f98660a9849dcba430c50369debd8: Deleting consensus metadata
I20260806 09:33:50.145457 17377 catalog_manager.cc:4841] TS 126f98660a9849dcba430c50369debd8 (127.20.113.2:39097): tablet cbd4bc32e4c342c1bb978f62bec5ee08 (table test-table [id=0acd2b20aa8148739a17e90c337b7f5e]) successfully deleted
I20260806 09:33:50.149406 18129 ts_tablet_manager.cc:1939] T cbd4bc32e4c342c1bb978f62bec5ee08 P b51d72485b634e3795173bd1c98077bc: Deleting consensus metadata
I20260806 09:33:50.151242 17379 catalog_manager.cc:4841] TS b51d72485b634e3795173bd1c98077bc (127.20.113.3:43555): tablet cbd4bc32e4c342c1bb978f62bec5ee08 (table test-table [id=0acd2b20aa8148739a17e90c337b7f5e]) successfully deleted
I20260806 09:33:50.245577 18130 tablet_replica.cc:330] T cbd4bc32e4c342c1bb978f62bec5ee08 P 0f53e3e20c9c4ecba50ba7034fbd8839: stopping tablet replica
I20260806 09:33:50.246055 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 17490
I20260806 09:33:50.275476 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 17683
W20260806 09:33:50.275564 17379 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.20.113.1:45017 (error 108)
I20260806 09:33:50.277505 18136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:50.276809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.1:45017 (local address 127.0.0.1:33016)
0806 09:33:50.277238 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:50.277348 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.1:45017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":222,"threads_started":1}
W20260806 09:33:50.277827 17379 catalog_manager.cc:4584] TS 0f53e3e20c9c4ecba50ba7034fbd8839 (127.20.113.1:45017): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet cbd4bc32e4c342c1bb978f62bec5ee08: Network error: Client connection negotiation failed: client connection to 127.20.113.1:45017: connect: Connection refused (error 111)
I20260806 09:33:50.284448 18091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:50.277128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34994 (local address 127.20.113.2:39097)
0806 09:33:50.277315 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0806 09:33:50.277323 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 09:33:50.277341 (+ 18us) server_negotiation.cc:407] Connection header received
0806 09:33:50.277421 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:33:50.277426 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:33:50.277554 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:33:50.277657 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:33:50.278346 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:50.279184 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:50.281123 (+ 1939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:50.281542 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:50.281646 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:50.282887 (+ 1241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:33:50.282895 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:33:50.282910 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:33:50.283499 (+ 589us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:33:50.283545 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:33:50.283565 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:33:50.283650 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 09:33:50.283887 (+ 237us) server_negotiation.cc:299] Negotiation successful
0806 09:33:50.283985 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 09:33:50.285158 17860 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:33:50.331550 18136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:50.330947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.1:45017 (local address 127.0.0.1:33022)
0806 09:33:50.331218 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:50.331334 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.1:45017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
I20260806 09:33:50.332903 18136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:50.332437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.1:45017 (local address 127.0.0.1:33028)
0806 09:33:50.332617 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:50.332707 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.1:45017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260806 09:33:50.376214 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 17683
I20260806 09:33:50.394006 18136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:50.393339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.1:45017 (local address 127.0.0.1:33040)
0806 09:33:50.393663 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:50.393786 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.1:45017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
I20260806 09:33:50.395323 18136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:50.394862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.1:45017 (local address 127.0.0.1:33052)
0806 09:33:50.395047 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:50.395147 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.1:45017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260806 09:33:50.403796 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 17872
I20260806 09:33:50.412137 18088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:50.404730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52844 (local address 127.20.113.3:43555)
0806 09:33:50.404907 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0806 09:33:50.404911 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 09:33:50.405041 (+ 130us) server_negotiation.cc:407] Connection header received
0806 09:33:50.405307 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:33:50.405312 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:33:50.405399 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:33:50.405506 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:33:50.406053 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:50.406894 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:50.408800 (+ 1906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:50.409260 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:50.409363 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:50.410604 (+ 1241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:33:50.410612 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:33:50.410628 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:33:50.411308 (+ 680us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:33:50.411358 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:33:50.411380 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:33:50.411474 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 09:33:50.411638 (+ 164us) server_negotiation.cc:299] Negotiation successful
0806 09:33:50.411717 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260806 09:33:50.412961 18060 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:33:50.475692 18136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:50.475156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.1:45017 (local address 127.0.0.1:33060)
0806 09:33:50.475395 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:50.475506 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.1:45017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260806 09:33:50.476881 18136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:50.476417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.1:45017 (local address 127.0.0.1:33062)
0806 09:33:50.476626 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:50.476718 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.1:45017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
I20260806 09:33:50.540840 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 17872
I20260806 09:33:50.570458 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 17350
I20260806 09:33:50.571583 17451 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:33:50.678268 18136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:50.677639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.1:45017 (local address 127.0.0.1:33066)
0806 09:33:50.677879 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0806 09:33:50.678018 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.1:45017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260806 09:33:50.678555 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 17350
2026-08-06T09:33:50Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (3112 ms)
[----------] 6 tests from EncryptionPolicyTest (23834 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786008495763054-20932-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 09:33:50 dist-test-slave-45dx krb5kdc[18146](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 09:33:50 dist-test-slave-45dx krb5kdc[18146](info): set up 2 sockets
Aug 06 09:33:50 dist-test-slave-45dx krb5kdc[18146](info): commencing operation
krb5kdc: starting...
W20260806 09:33:51.989403 20932 mini_kdc.cc:121] Time spent starting KDC: real 1.260s user 0.006s sys 0.048s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786008495763054-20932-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786008495763054-20932-0/krb5kdc/test-user.keytab.
Aug 06 09:33:52 dist-test-slave-45dx krb5kdc[18146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008832, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T09:33:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T09:33:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786008495763054-20932-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786008495763054-20932-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 09:33:52.325934 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786008495763054-20932-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786008495763054-20932-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786008495763054-20932-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37847
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786008495763054-20932-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37847
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:33:52.687847 18167 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:33:52.688129 18167 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:33:52.688391 18167 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:33:52.695358 18167 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 09:33:52.695453 18167 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:33:52.695575 18167 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:33:52.695686 18167 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 09:33:52.695749 18167 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 09:33:52.705287 18167 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786008495763054-20932-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786008495763054-20932-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37847
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786008495763054-20932-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37847
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786008495763054-20932-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:33:52.706496 18167 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:33:52.707785 18167 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:33:52.719347 18175 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:33:52.719491 18173 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:33:52.719478 18177 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:33:52.720602 18167 server_base.cc:1034] running on GCE node
I20260806 09:33:52.721312 18167 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:33:52.722630 18167 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:33:52.723866 18167 hybrid_clock.cc:648] HybridClock initialized: now 1786008832723833 us; error 43 us; skew 500 ppm
I20260806 09:33:52.724205 18167 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:33:52 dist-test-slave-45dx krb5kdc[18146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008832, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:33:52.727191 18167 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 09:33:52.728662 18167 webserver.cc:460] Webserver started at http://127.0.0.1:42907/ using document root <none> and password file <none>
I20260806 09:33:52.729324 18167 fs_manager.cc:359] Metadata directory not provided
I20260806 09:33:52.729489 18167 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:33:52.729796 18167 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:33:52.732846 18167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786008495763054-20932-0/minicluster-data/master-0/data/instance:
uuid: "60b4a9395b314978acf763a7f8ca5783"
format_stamp: "Formatted at 2026-08-06 09:33:52 on dist-test-slave-45dx"
server_key: "14f62aa55a6f5a7e7185bf835c20fd1e"
server_key_iv: "3bceaf8b4542d926ac01c31113840be4"
server_key_version: "encryptionkey@0"
I20260806 09:33:52.733598 18167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786008495763054-20932-0/minicluster-data/master-0/wal/instance:
uuid: "60b4a9395b314978acf763a7f8ca5783"
format_stamp: "Formatted at 2026-08-06 09:33:52 on dist-test-slave-45dx"
server_key: "14f62aa55a6f5a7e7185bf835c20fd1e"
server_key_iv: "3bceaf8b4542d926ac01c31113840be4"
server_key_version: "encryptionkey@0"
I20260806 09:33:52.738620 18167 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 09:33:52.742120 18187 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:33:52.743422 18167 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 09:33:52.743604 18167 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786008495763054-20932-0/minicluster-data/master-0/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786008495763054-20932-0/minicluster-data/master-0/wal
uuid: "60b4a9395b314978acf763a7f8ca5783"
format_stamp: "Formatted at 2026-08-06 09:33:52 on dist-test-slave-45dx"
server_key: "14f62aa55a6f5a7e7185bf835c20fd1e"
server_key_iv: "3bceaf8b4542d926ac01c31113840be4"
server_key_version: "encryptionkey@0"
I20260806 09:33:52.743799 18167 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786008495763054-20932-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786008495763054-20932-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786008495763054-20932-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:33:52.761965 18167 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:33:52.765693 18167 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:33:52.765973 18167 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:33:52.784389 18167 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37847
I20260806 09:33:52.784480 18265 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37847 every 8 connection(s)
I20260806 09:33:52.785728 18167 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
I20260806 09:33:52.789309 18266 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:33:52.791522 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 18167
I20260806 09:33:52.791738 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786008495763054-20932-0/minicluster-data/master-0/wal/instance
I20260806 09:33:52.792460 20932 external_mini_cluster.cc:1428] Setting key 3edc008f704570545baf95a9760ad734
I20260806 09:33:52.801074 18266 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60b4a9395b314978acf763a7f8ca5783: Bootstrap starting.
I20260806 09:33:52.804695 18266 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 60b4a9395b314978acf763a7f8ca5783: Neither blocks nor log segments found. Creating new log.
I20260806 09:33:52.806063 18266 log.cc:824] T 00000000000000000000000000000000 P 60b4a9395b314978acf763a7f8ca5783: Log is configured to *not* fsync() on all Append() calls
Aug 06 09:33:52 dist-test-slave-45dx krb5kdc[18146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008832, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 09:33:52.808970 18266 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60b4a9395b314978acf763a7f8ca5783: No bootstrap required, opened a new log
I20260806 09:33:52.815613 18270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:52.794318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36259 (local address 127.0.0.1:37847)
0806 09:33:52.795196 (+ 878us) server_negotiation.cc:206] Beginning negotiation
0806 09:33:52.795206 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 09:33:52.795245 (+ 39us) server_negotiation.cc:407] Connection header received
0806 09:33:52.796981 (+ 1736us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:33:52.797003 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:33:52.797367 (+ 364us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:33:52.797673 (+ 306us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:33:52.799403 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:52.800683 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:52.802452 (+ 1769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:52.803079 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:52.803195 (+ 116us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:52.808456 (+ 5261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:33:52.808464 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:33:52.808477 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:33:52.808512 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:33:52.811532 (+ 3020us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:33:52.812347 (+ 815us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:33:52.812363 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:33:52.812371 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:33:52.812547 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:33:52.813037 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:33:52.813041 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:33:52.813042 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:33:52.813552 (+ 510us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:33:52.813901 (+ 349us) server_negotiation.cc:1035] Waiting for connection context
0806 09:33:52.814402 (+ 501us) server_negotiation.cc:299] Negotiation successful
0806 09:33:52.814690 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":675,"thread_start_us":462,"threads_started":1}
I20260806 09:33:52.816614 18266 raft_consensus.cc:348] T 00000000000000000000000000000000 P 60b4a9395b314978acf763a7f8ca5783 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60b4a9395b314978acf763a7f8ca5783" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37847 } }
I20260806 09:33:52.817028 18266 raft_consensus.cc:374] T 00000000000000000000000000000000 P 60b4a9395b314978acf763a7f8ca5783 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:33:52.817142 18266 raft_consensus.cc:729] T 00000000000000000000000000000000 P 60b4a9395b314978acf763a7f8ca5783 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60b4a9395b314978acf763a7f8ca5783, State: Initialized, Role: FOLLOWER
I20260806 09:33:52.817749 18266 consensus_queue.cc:260] T 00000000000000000000000000000000 P 60b4a9395b314978acf763a7f8ca5783 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60b4a9395b314978acf763a7f8ca5783" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37847 } }
I20260806 09:33:52.818054 18266 raft_consensus.cc:388] T 00000000000000000000000000000000 P 60b4a9395b314978acf763a7f8ca5783 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 09:33:52.818228 18266 raft_consensus.cc:482] T 00000000000000000000000000000000 P 60b4a9395b314978acf763a7f8ca5783 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 09:33:52.818413 18266 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 60b4a9395b314978acf763a7f8ca5783 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:33:52.819861 20932 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 09:33:52.821303 18266 raft_consensus.cc:504] T 00000000000000000000000000000000 P 60b4a9395b314978acf763a7f8ca5783 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60b4a9395b314978acf763a7f8ca5783" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37847 } }
I20260806 09:33:52.821834 18266 leader_election.cc:302] T 00000000000000000000000000000000 P 60b4a9395b314978acf763a7f8ca5783 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 60b4a9395b314978acf763a7f8ca5783; no voters:
I20260806 09:33:52.822438 18266 leader_election.cc:288] T 00000000000000000000000000000000 P 60b4a9395b314978acf763a7f8ca5783 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 09:33:52.822510 18274 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 60b4a9395b314978acf763a7f8ca5783 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:33:52.824856 18266 sys_catalog.cc:564] T 00000000000000000000000000000000 P 60b4a9395b314978acf763a7f8ca5783 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 09:33:52.834650 18274 raft_consensus.cc:686] T 00000000000000000000000000000000 P 60b4a9395b314978acf763a7f8ca5783 [term 1 LEADER]: Becoming Leader. State: Replica: 60b4a9395b314978acf763a7f8ca5783, State: Running, Role: LEADER
I20260806 09:33:52.835546 18274 consensus_queue.cc:237] T 00000000000000000000000000000000 P 60b4a9395b314978acf763a7f8ca5783 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60b4a9395b314978acf763a7f8ca5783" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37847 } }
I20260806 09:33:52.840035 18275 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60b4a9395b314978acf763a7f8ca5783 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "60b4a9395b314978acf763a7f8ca5783" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60b4a9395b314978acf763a7f8ca5783" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37847 } } }
I20260806 09:33:52.840410 18275 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60b4a9395b314978acf763a7f8ca5783 [sys.catalog]: This master's current role is: LEADER
I20260806 09:33:52.840914 18276 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60b4a9395b314978acf763a7f8ca5783 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 60b4a9395b314978acf763a7f8ca5783. Latest consensus state: current_term: 1 leader_uuid: "60b4a9395b314978acf763a7f8ca5783" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60b4a9395b314978acf763a7f8ca5783" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37847 } } }
I20260806 09:33:52.841514 18276 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60b4a9395b314978acf763a7f8ca5783 [sys.catalog]: This master's current role is: LEADER
W20260806 09:33:52.845539 18304 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 60b4a9395b314978acf763a7f8ca5783: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 09:33:52.845726 18304 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 09:33:52.845932 18305 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 09:33:52.848462 18305 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 09:33:52.853721 18270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:52.833642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50507 (local address 127.0.0.1:37847)
0806 09:33:52.833827 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0806 09:33:52.833835 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 09:33:52.834726 (+ 891us) server_negotiation.cc:407] Connection header received
0806 09:33:52.835021 (+ 295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:33:52.835027 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:33:52.835103 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:33:52.835186 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:33:52.837712 (+ 2526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:52.839493 (+ 1781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:52.841470 (+ 1977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:52.842119 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:52.842238 (+ 119us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:52.845793 (+ 3555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:33:52.845803 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:33:52.845807 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:33:52.845848 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:33:52.849892 (+ 4044us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:33:52.850893 (+ 1001us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:33:52.850899 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:33:52.850902 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:33:52.851046 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:33:52.851642 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:33:52.851648 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:33:52.851651 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:33:52.852087 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:33:52.852395 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0806 09:33:52.853172 (+ 777us) server_negotiation.cc:299] Negotiation successful
0806 09:33:52.853364 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 09:33:52.858579 18305 catalog_manager.cc:1347] Generated new cluster ID: 37e8bfe109e54c9ab188be358dd41d9c
I20260806 09:33:52.858695 18305 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 09:33:52.873577 18305 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 09:33:52.874812 18305 catalog_manager.cc:1504] Loading token signing keys...
I20260806 09:33:52.885761 18305 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 60b4a9395b314978acf763a7f8ca5783: Generated new TSK 0
I20260806 09:33:52.886847 18305 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 09:33:52.901494 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 18167
I20260806 09:33:52.902817 18257 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:33:52.994922 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 18167
W20260806 09:33:53.020135 18144 connection.cc:535] client connection to 127.0.0.1:37847 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37847): Connection reset by peer (error 104)
2026-08-06T09:33:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (2310 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786008495763054-20932-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 09:33:53 dist-test-slave-45dx krb5kdc[18313](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 09:33:53 dist-test-slave-45dx krb5kdc[18313](info): set up 2 sockets
Aug 06 09:33:53 dist-test-slave-45dx krb5kdc[18313](info): commencing operation
krb5kdc: starting...
W20260806 09:33:54.298041 20932 mini_kdc.cc:121] Time spent starting KDC: real 1.259s user 0.004s sys 0.051s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786008495763054-20932-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786008495763054-20932-0/krb5kdc/test-user.keytab.
Aug 06 09:33:54 dist-test-slave-45dx krb5kdc[18313](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008834, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T09:33:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T09:33:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786008495763054-20932-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786008495763054-20932-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 09:33:54.654991 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786008495763054-20932-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786008495763054-20932-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786008495763054-20932-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46571
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786008495763054-20932-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46571
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:33:54.998677 18337 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:33:54.999002 18337 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:33:54.999178 18337 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:33:55.006227 18337 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 09:33:55.006388 18337 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:33:55.006522 18337 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:33:55.006635 18337 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 09:33:55.006726 18337 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 09:33:55.016157 18337 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786008495763054-20932-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786008495763054-20932-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46571
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786008495763054-20932-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46571
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786008495763054-20932-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:33:55.017453 18337 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:33:55.018785 18337 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:33:55.030686 18343 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:33:55.030650 18344 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:33:55.030898 18346 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:33:55.031629 18337 server_base.cc:1034] running on GCE node
I20260806 09:33:55.032485 18337 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:33:55.033829 18337 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:33:55.035095 18337 hybrid_clock.cc:648] HybridClock initialized: now 1786008835035061 us; error 40 us; skew 500 ppm
I20260806 09:33:55.035496 18337 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:33:55 dist-test-slave-45dx krb5kdc[18313](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008835, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:33:55.038666 18337 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 09:33:55.040151 18337 webserver.cc:460] Webserver started at http://127.0.0.1:38073/ using document root <none> and password file <none>
I20260806 09:33:55.040858 18337 fs_manager.cc:359] Metadata directory not provided
I20260806 09:33:55.041004 18337 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:33:55.041312 18337 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:33:55.044243 18337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786008495763054-20932-0/minicluster-data/master-0/data/instance:
uuid: "4eaf68d3d9c648308da0572e89793e90"
format_stamp: "Formatted at 2026-08-06 09:33:55 on dist-test-slave-45dx"
server_key: "6d199a84ad0793e7ab1306ed1475930d"
server_key_iv: "3980e00b1b214aac19f98d49c1570024"
server_key_version: "encryptionkey@0"
I20260806 09:33:55.045060 18337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786008495763054-20932-0/minicluster-data/master-0/wal/instance:
uuid: "4eaf68d3d9c648308da0572e89793e90"
format_stamp: "Formatted at 2026-08-06 09:33:55 on dist-test-slave-45dx"
server_key: "6d199a84ad0793e7ab1306ed1475930d"
server_key_iv: "3980e00b1b214aac19f98d49c1570024"
server_key_version: "encryptionkey@0"
I20260806 09:33:55.050040 18337 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 09:33:55.053380 18356 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:33:55.054616 18337 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 09:33:55.054800 18337 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786008495763054-20932-0/minicluster-data/master-0/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786008495763054-20932-0/minicluster-data/master-0/wal
uuid: "4eaf68d3d9c648308da0572e89793e90"
format_stamp: "Formatted at 2026-08-06 09:33:55 on dist-test-slave-45dx"
server_key: "6d199a84ad0793e7ab1306ed1475930d"
server_key_iv: "3980e00b1b214aac19f98d49c1570024"
server_key_version: "encryptionkey@0"
I20260806 09:33:55.054991 18337 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786008495763054-20932-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786008495763054-20932-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786008495763054-20932-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:33:55.071234 18337 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:33:55.074990 18337 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:33:55.075268 18337 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:33:55.093995 18337 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46571
I20260806 09:33:55.094030 18429 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46571 every 8 connection(s)
I20260806 09:33:55.095345 18337 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
I20260806 09:33:55.098199 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 18337
I20260806 09:33:55.098419 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786008495763054-20932-0/minicluster-data/master-0/wal/instance
I20260806 09:33:55.098922 20932 external_mini_cluster.cc:1428] Setting key 4733b0ae872db9cd81392cc73e5fb927
I20260806 09:33:55.099750 18433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:33:55.113766 18436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:55.100846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46611 (local address 127.0.0.1:46571)
0806 09:33:55.101652 (+ 806us) server_negotiation.cc:206] Beginning negotiation
0806 09:33:55.101668 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 09:33:55.101716 (+ 48us) server_negotiation.cc:407] Connection header received
0806 09:33:55.104124 (+ 2408us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:33:55.104153 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:33:55.104687 (+ 534us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:33:55.105135 (+ 448us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:33:55.106260 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:55.107590 (+ 1330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:55.109496 (+ 1906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:55.110120 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:55.110244 (+ 124us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:55.111538 (+ 1294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:33:55.111545 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:33:55.111557 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:33:55.112116 (+ 559us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:33:55.112186 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:33:55.112210 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:33:55.112423 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0806 09:33:55.112676 (+ 253us) server_negotiation.cc:299] Negotiation successful
0806 09:33:55.112849 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":548,"thread_start_us":334,"threads_started":1}
I20260806 09:33:55.114534 18433 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4eaf68d3d9c648308da0572e89793e90: Bootstrap starting.
I20260806 09:33:55.117411 20932 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 09:33:55.117754 18433 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4eaf68d3d9c648308da0572e89793e90: Neither blocks nor log segments found. Creating new log.
I20260806 09:33:55.118963 18433 log.cc:824] T 00000000000000000000000000000000 P 4eaf68d3d9c648308da0572e89793e90: Log is configured to *not* fsync() on all Append() calls
I20260806 09:33:55.122241 18433 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4eaf68d3d9c648308da0572e89793e90: No bootstrap required, opened a new log
I20260806 09:33:55.131539 18433 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4eaf68d3d9c648308da0572e89793e90 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4eaf68d3d9c648308da0572e89793e90" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46571 } }
I20260806 09:33:55.131932 18433 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4eaf68d3d9c648308da0572e89793e90 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:33:55.132071 18433 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4eaf68d3d9c648308da0572e89793e90 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4eaf68d3d9c648308da0572e89793e90, State: Initialized, Role: FOLLOWER
I20260806 09:33:55.132694 18433 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4eaf68d3d9c648308da0572e89793e90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4eaf68d3d9c648308da0572e89793e90" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46571 } }
I20260806 09:33:55.132932 18433 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4eaf68d3d9c648308da0572e89793e90 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 09:33:55.133076 18433 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4eaf68d3d9c648308da0572e89793e90 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 09:33:55.133255 18433 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4eaf68d3d9c648308da0572e89793e90 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:33:55.134838 18436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:55.122739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44181 (local address 127.0.0.1:46571)
0806 09:33:55.122973 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0806 09:33:55.122980 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:33:55.123345 (+ 365us) server_negotiation.cc:407] Connection header received
0806 09:33:55.123519 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:33:55.123522 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:33:55.123575 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:33:55.123652 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:33:55.124282 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:55.125112 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:55.127119 (+ 2007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:55.127594 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:55.127687 (+ 93us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:55.133924 (+ 6237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:33:55.133929 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:33:55.133932 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:33:55.133950 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:33:55.134004 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:33:55.134022 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:33:55.134109 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 09:33:55.134372 (+ 263us) server_negotiation.cc:299] Negotiation successful
0806 09:33:55.134443 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 09:33:55.136061 18433 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4eaf68d3d9c648308da0572e89793e90 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4eaf68d3d9c648308da0572e89793e90" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46571 } }
I20260806 09:33:55.136579 18433 leader_election.cc:302] T 00000000000000000000000000000000 P 4eaf68d3d9c648308da0572e89793e90 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4eaf68d3d9c648308da0572e89793e90; no voters:
I20260806 09:33:55.137131 18433 leader_election.cc:288] T 00000000000000000000000000000000 P 4eaf68d3d9c648308da0572e89793e90 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 09:33:55.137257 18450 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4eaf68d3d9c648308da0572e89793e90 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:33:55.138072 18450 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4eaf68d3d9c648308da0572e89793e90 [term 1 LEADER]: Becoming Leader. State: Replica: 4eaf68d3d9c648308da0572e89793e90, State: Running, Role: LEADER
I20260806 09:33:55.138691 18450 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4eaf68d3d9c648308da0572e89793e90 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4eaf68d3d9c648308da0572e89793e90" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46571 } }
I20260806 09:33:55.139346 18433 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4eaf68d3d9c648308da0572e89793e90 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 09:33:55.141587 18452 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4eaf68d3d9c648308da0572e89793e90 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4eaf68d3d9c648308da0572e89793e90. Latest consensus state: current_term: 1 leader_uuid: "4eaf68d3d9c648308da0572e89793e90" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4eaf68d3d9c648308da0572e89793e90" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46571 } } }
I20260806 09:33:55.142061 18452 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4eaf68d3d9c648308da0572e89793e90 [sys.catalog]: This master's current role is: LEADER
I20260806 09:33:55.143821 18464 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 09:33:55.143795 18451 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4eaf68d3d9c648308da0572e89793e90 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4eaf68d3d9c648308da0572e89793e90" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4eaf68d3d9c648308da0572e89793e90" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46571 } } }
I20260806 09:33:55.144196 18451 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4eaf68d3d9c648308da0572e89793e90 [sys.catalog]: This master's current role is: LEADER
I20260806 09:33:55.149184 18464 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 09:33:55.158310 18464 catalog_manager.cc:1347] Generated new cluster ID: ae7127c87c304846904b149a88d137a1
I20260806 09:33:55.158471 18464 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 09:33:55.187526 18464 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 09:33:55.189172 18464 catalog_manager.cc:1504] Loading token signing keys...
I20260806 09:33:55.199028 18464 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4eaf68d3d9c648308da0572e89793e90: Generated new TSK 0
I20260806 09:33:55.199842 18464 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 09:33:55.217629 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 18337
W20260806 09:33:55.218756 18423 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:46611
W20260806 09:33:55.219700 20932 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 18337: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 09:33:55.219900 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 18337
W20260806 09:33:55.245687 18310 connection.cc:535] client connection to 127.0.0.1:46571 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46571): Connection reset by peer (error 104)
2026-08-06T09:33:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (2225 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786008495763054-20932-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 09:33:55 dist-test-slave-45dx krb5kdc[18486](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 09:33:55 dist-test-slave-45dx krb5kdc[18486](info): set up 2 sockets
Aug 06 09:33:55 dist-test-slave-45dx krb5kdc[18486](info): commencing operation
krb5kdc: starting...
W20260806 09:33:56.541142 20932 mini_kdc.cc:121] Time spent starting KDC: real 1.276s user 0.000s sys 0.057s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786008495763054-20932-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786008495763054-20932-0/krb5kdc/test-user.keytab.
Aug 06 09:33:56 dist-test-slave-45dx krb5kdc[18486](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008836, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T09:33:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T09:33:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786008495763054-20932-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786008495763054-20932-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 09:33:56.887694 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786008495763054-20932-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786008495763054-20932-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786008495763054-20932-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38573
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786008495763054-20932-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38573
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:33:57.232174 18510 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:33:57.232545 18510 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:33:57.232717 18510 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:33:57.240559 18510 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 09:33:57.240696 18510 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:33:57.240780 18510 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:33:57.240882 18510 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 09:33:57.240948 18510 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 09:33:57.250640 18510 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786008495763054-20932-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786008495763054-20932-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38573
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786008495763054-20932-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38573
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786008495763054-20932-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:33:57.251961 18510 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:33:57.253274 18510 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:33:57.265730 18517 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:33:57.265806 18518 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:33:57.265826 18520 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:33:57.266736 18510 server_base.cc:1034] running on GCE node
I20260806 09:33:57.267513 18510 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:33:57.268935 18510 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:33:57.270172 18510 hybrid_clock.cc:648] HybridClock initialized: now 1786008837270141 us; error 46 us; skew 500 ppm
I20260806 09:33:57.270514 18510 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:33:57 dist-test-slave-45dx krb5kdc[18486](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008837, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:33:57.273722 18510 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 09:33:57.275146 18510 webserver.cc:460] Webserver started at http://127.0.0.1:33521/ using document root <none> and password file <none>
I20260806 09:33:57.275770 18510 fs_manager.cc:359] Metadata directory not provided
I20260806 09:33:57.275918 18510 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:33:57.276202 18510 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:33:57.279050 18510 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786008495763054-20932-0/minicluster-data/master-0/data/instance:
uuid: "f23979ba10d44654bd80f59e6fe27bb4"
format_stamp: "Formatted at 2026-08-06 09:33:57 on dist-test-slave-45dx"
server_key: "6f99618d5d45944abb834bd663cc353c"
server_key_iv: "bcd0b94a4f8460eff2a0cbe6a7532fd3"
server_key_version: "encryptionkey@0"
I20260806 09:33:57.279852 18510 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786008495763054-20932-0/minicluster-data/master-0/wal/instance:
uuid: "f23979ba10d44654bd80f59e6fe27bb4"
format_stamp: "Formatted at 2026-08-06 09:33:57 on dist-test-slave-45dx"
server_key: "6f99618d5d45944abb834bd663cc353c"
server_key_iv: "bcd0b94a4f8460eff2a0cbe6a7532fd3"
server_key_version: "encryptionkey@0"
I20260806 09:33:57.284835 18510 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 09:33:57.288242 18530 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:33:57.289649 18510 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 09:33:57.289865 18510 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786008495763054-20932-0/minicluster-data/master-0/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786008495763054-20932-0/minicluster-data/master-0/wal
uuid: "f23979ba10d44654bd80f59e6fe27bb4"
format_stamp: "Formatted at 2026-08-06 09:33:57 on dist-test-slave-45dx"
server_key: "6f99618d5d45944abb834bd663cc353c"
server_key_iv: "bcd0b94a4f8460eff2a0cbe6a7532fd3"
server_key_version: "encryptionkey@0"
I20260806 09:33:57.290076 18510 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786008495763054-20932-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786008495763054-20932-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786008495763054-20932-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:33:57.295688 18510 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:33:57.299692 18510 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:33:57.300042 18510 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:33:57.318755 18510 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38573
I20260806 09:33:57.318846 18616 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38573 every 8 connection(s)
I20260806 09:33:57.320185 18510 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
I20260806 09:33:57.323863 18617 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:33:57.329753 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 18510
I20260806 09:33:57.330003 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786008495763054-20932-0/minicluster-data/master-0/wal/instance
I20260806 09:33:57.330739 20932 external_mini_cluster.cc:1428] Setting key 45b34ba7776fbe6091a961fc49e61f16
I20260806 09:33:57.335611 18617 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f23979ba10d44654bd80f59e6fe27bb4: Bootstrap starting.
I20260806 09:33:57.338761 18624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:57.332681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47359 (local address 127.0.0.1:38573)
0806 09:33:57.333577 (+ 896us) server_negotiation.cc:206] Beginning negotiation
0806 09:33:57.333588 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 09:33:57.333631 (+ 43us) server_negotiation.cc:407] Connection header received
0806 09:33:57.335249 (+ 1618us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:33:57.335273 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:33:57.335582 (+ 309us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:33:57.335966 (+ 384us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:33:57.337189 (+ 1223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:33:57.337195 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:33:57.337207 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:33:57.337220 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:33:57.337299 (+ 79us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:33:57.337319 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:33:57.337488 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0806 09:33:57.337657 (+ 169us) server_negotiation.cc:299] Negotiation successful
0806 09:33:57.337893 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":656,"thread_start_us":439,"threads_started":1}
I20260806 09:33:57.339560 18617 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f23979ba10d44654bd80f59e6fe27bb4: Neither blocks nor log segments found. Creating new log.
I20260806 09:33:57.340764 18617 log.cc:824] T 00000000000000000000000000000000 P f23979ba10d44654bd80f59e6fe27bb4: Log is configured to *not* fsync() on all Append() calls
I20260806 09:33:57.342846 20932 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 09:33:57.344241 18617 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f23979ba10d44654bd80f59e6fe27bb4: No bootstrap required, opened a new log
I20260806 09:33:57.350577 18624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:57.348121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49037 (local address 127.0.0.1:38573)
0806 09:33:57.348339 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0806 09:33:57.348346 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:33:57.348718 (+ 372us) server_negotiation.cc:407] Connection header received
0806 09:33:57.348864 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:33:57.348869 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:33:57.348925 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:33:57.349027 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:33:57.349953 (+ 926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:33:57.349959 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:33:57.349963 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:33:57.349989 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:33:57.350045 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:33:57.350066 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:33:57.350195 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0806 09:33:57.350244 (+ 49us) server_negotiation.cc:299] Negotiation successful
0806 09:33:57.350305 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 09:33:57.353716 18617 raft_consensus.cc:348] T 00000000000000000000000000000000 P f23979ba10d44654bd80f59e6fe27bb4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f23979ba10d44654bd80f59e6fe27bb4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38573 } }
I20260806 09:33:57.354071 18617 raft_consensus.cc:374] T 00000000000000000000000000000000 P f23979ba10d44654bd80f59e6fe27bb4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:33:57.354166 18617 raft_consensus.cc:729] T 00000000000000000000000000000000 P f23979ba10d44654bd80f59e6fe27bb4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f23979ba10d44654bd80f59e6fe27bb4, State: Initialized, Role: FOLLOWER
I20260806 09:33:57.354666 18617 consensus_queue.cc:260] T 00000000000000000000000000000000 P f23979ba10d44654bd80f59e6fe27bb4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f23979ba10d44654bd80f59e6fe27bb4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38573 } }
I20260806 09:33:57.354882 18617 raft_consensus.cc:388] T 00000000000000000000000000000000 P f23979ba10d44654bd80f59e6fe27bb4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 09:33:57.354980 18617 raft_consensus.cc:482] T 00000000000000000000000000000000 P f23979ba10d44654bd80f59e6fe27bb4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 09:33:57.355154 18617 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f23979ba10d44654bd80f59e6fe27bb4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:33:57.357741 18617 raft_consensus.cc:504] T 00000000000000000000000000000000 P f23979ba10d44654bd80f59e6fe27bb4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f23979ba10d44654bd80f59e6fe27bb4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38573 } }
I20260806 09:33:57.358227 18617 leader_election.cc:302] T 00000000000000000000000000000000 P f23979ba10d44654bd80f59e6fe27bb4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f23979ba10d44654bd80f59e6fe27bb4; no voters:
I20260806 09:33:57.358846 18617 leader_election.cc:288] T 00000000000000000000000000000000 P f23979ba10d44654bd80f59e6fe27bb4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 09:33:57.359018 18636 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f23979ba10d44654bd80f59e6fe27bb4 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:33:57.359751 18636 raft_consensus.cc:686] T 00000000000000000000000000000000 P f23979ba10d44654bd80f59e6fe27bb4 [term 1 LEADER]: Becoming Leader. State: Replica: f23979ba10d44654bd80f59e6fe27bb4, State: Running, Role: LEADER
I20260806 09:33:57.360276 18636 consensus_queue.cc:237] T 00000000000000000000000000000000 P f23979ba10d44654bd80f59e6fe27bb4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f23979ba10d44654bd80f59e6fe27bb4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38573 } }
I20260806 09:33:57.361055 18617 sys_catalog.cc:564] T 00000000000000000000000000000000 P f23979ba10d44654bd80f59e6fe27bb4 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 09:33:57.363974 18637 sys_catalog.cc:455] T 00000000000000000000000000000000 P f23979ba10d44654bd80f59e6fe27bb4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f23979ba10d44654bd80f59e6fe27bb4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f23979ba10d44654bd80f59e6fe27bb4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38573 } } }
I20260806 09:33:57.364444 18637 sys_catalog.cc:458] T 00000000000000000000000000000000 P f23979ba10d44654bd80f59e6fe27bb4 [sys.catalog]: This master's current role is: LEADER
I20260806 09:33:57.364547 18638 sys_catalog.cc:455] T 00000000000000000000000000000000 P f23979ba10d44654bd80f59e6fe27bb4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f23979ba10d44654bd80f59e6fe27bb4. Latest consensus state: current_term: 1 leader_uuid: "f23979ba10d44654bd80f59e6fe27bb4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f23979ba10d44654bd80f59e6fe27bb4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38573 } } }
I20260806 09:33:57.365007 18638 sys_catalog.cc:458] T 00000000000000000000000000000000 P f23979ba10d44654bd80f59e6fe27bb4 [sys.catalog]: This master's current role is: LEADER
I20260806 09:33:57.367059 18647 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 09:33:57.372100 18647 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 09:33:57.380475 18647 catalog_manager.cc:1347] Generated new cluster ID: f4bb3e2ff19c4bcfa9465d0f63763923
I20260806 09:33:57.380584 18647 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 09:33:57.401870 18647 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 09:33:57.402302 18647 catalog_manager.cc:1504] Loading token signing keys...
I20260806 09:33:57.407553 18647 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f23979ba10d44654bd80f59e6fe27bb4: Generated new TSK 0
I20260806 09:33:57.408273 18647 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 09:33:57.414395 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 18510
W20260806 09:33:57.415474 18610 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:47359
W20260806 09:33:57.416172 20932 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 18510: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 09:33:57.416359 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 18510
2026-08-06T09:33:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (2197 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786008495763054-20932-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 09:33:57 dist-test-slave-45dx krb5kdc[18672](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 09:33:57 dist-test-slave-45dx krb5kdc[18672](info): set up 2 sockets
Aug 06 09:33:57 dist-test-slave-45dx krb5kdc[18672](info): commencing operation
krb5kdc: starting...
W20260806 09:33:58.744848 20932 mini_kdc.cc:121] Time spent starting KDC: real 1.282s user 0.008s sys 0.047s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786008495763054-20932-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786008495763054-20932-0/krb5kdc/test-user.keytab.
Aug 06 09:33:58 dist-test-slave-45dx krb5kdc[18672](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008838, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T09:33:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T09:33:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786008495763054-20932-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786008495763054-20932-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 09:33:59.113377 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786008495763054-20932-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786008495763054-20932-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786008495763054-20932-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35427
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786008495763054-20932-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35427
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:33:59.471521 18696 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:33:59.471833 18696 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:33:59.471930 18696 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:33:59.479310 18696 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 09:33:59.479424 18696 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:33:59.479470 18696 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:33:59.479523 18696 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 09:33:59.479568 18696 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 09:33:59.489632 18696 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786008495763054-20932-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786008495763054-20932-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35427
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786008495763054-20932-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35427
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786008495763054-20932-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:33:59.490909 18696 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:33:59.492197 18696 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:33:59.504158 18701 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:33:59.504201 18706 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:33:59.504228 18710 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:33:59.505344 18696 server_base.cc:1034] running on GCE node
I20260806 09:33:59.506198 18696 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:33:59.507627 18696 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:33:59.508872 18696 hybrid_clock.cc:648] HybridClock initialized: now 1786008839508825 us; error 36 us; skew 500 ppm
I20260806 09:33:59.509274 18696 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:33:59 dist-test-slave-45dx krb5kdc[18672](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008839, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:33:59.512655 18696 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 09:33:59.514160 18696 webserver.cc:460] Webserver started at http://127.0.0.1:35583/ using document root <none> and password file <none>
I20260806 09:33:59.514855 18696 fs_manager.cc:359] Metadata directory not provided
I20260806 09:33:59.514976 18696 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:33:59.515231 18696 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:33:59.518342 18696 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786008495763054-20932-0/minicluster-data/master-0/data/instance:
uuid: "d1b9e01f1698456498b69ffbb2fa83bc"
format_stamp: "Formatted at 2026-08-06 09:33:59 on dist-test-slave-45dx"
server_key: "653f7b5f33e7538628e19639dbc51ca2"
server_key_iv: "8b0707fd0b1f1f222d5eddc8343290a4"
server_key_version: "encryptionkey@0"
I20260806 09:33:59.519114 18696 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786008495763054-20932-0/minicluster-data/master-0/wal/instance:
uuid: "d1b9e01f1698456498b69ffbb2fa83bc"
format_stamp: "Formatted at 2026-08-06 09:33:59 on dist-test-slave-45dx"
server_key: "653f7b5f33e7538628e19639dbc51ca2"
server_key_iv: "8b0707fd0b1f1f222d5eddc8343290a4"
server_key_version: "encryptionkey@0"
I20260806 09:33:59.524607 18696 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260806 09:33:59.528198 18718 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:33:59.529752 18696 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 09:33:59.529947 18696 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786008495763054-20932-0/minicluster-data/master-0/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786008495763054-20932-0/minicluster-data/master-0/wal
uuid: "d1b9e01f1698456498b69ffbb2fa83bc"
format_stamp: "Formatted at 2026-08-06 09:33:59 on dist-test-slave-45dx"
server_key: "653f7b5f33e7538628e19639dbc51ca2"
server_key_iv: "8b0707fd0b1f1f222d5eddc8343290a4"
server_key_version: "encryptionkey@0"
I20260806 09:33:59.530139 18696 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786008495763054-20932-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786008495763054-20932-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786008495763054-20932-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:33:59.545749 18696 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:33:59.550529 18696 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:33:59.550829 18696 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:33:59.570365 18696 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35427
I20260806 09:33:59.570379 18810 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35427 every 8 connection(s)
I20260806 09:33:59.571794 18696 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
I20260806 09:33:59.575688 18811 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:33:59.580731 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 18696
I20260806 09:33:59.580926 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786008495763054-20932-0/minicluster-data/master-0/wal/instance
I20260806 09:33:59.581373 20932 external_mini_cluster.cc:1428] Setting key 4f15517519cd79ac02cbbc13f1ef3688
I20260806 09:33:59.587344 18811 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1b9e01f1698456498b69ffbb2fa83bc: Bootstrap starting.
I20260806 09:33:59.591209 18811 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d1b9e01f1698456498b69ffbb2fa83bc: Neither blocks nor log segments found. Creating new log.
I20260806 09:33:59.592494 18811 log.cc:824] T 00000000000000000000000000000000 P d1b9e01f1698456498b69ffbb2fa83bc: Log is configured to *not* fsync() on all Append() calls
I20260806 09:33:59.596057 18811 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1b9e01f1698456498b69ffbb2fa83bc: No bootstrap required, opened a new log
Aug 06 09:33:59 dist-test-slave-45dx krb5kdc[18672](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008838, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 09:33:59.604642 18811 raft_consensus.cc:348] T 00000000000000000000000000000000 P d1b9e01f1698456498b69ffbb2fa83bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1b9e01f1698456498b69ffbb2fa83bc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35427 } }
I20260806 09:33:59.605042 18811 raft_consensus.cc:374] T 00000000000000000000000000000000 P d1b9e01f1698456498b69ffbb2fa83bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:33:59.605188 18811 raft_consensus.cc:729] T 00000000000000000000000000000000 P d1b9e01f1698456498b69ffbb2fa83bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1b9e01f1698456498b69ffbb2fa83bc, State: Initialized, Role: FOLLOWER
I20260806 09:33:59.605782 18811 consensus_queue.cc:260] T 00000000000000000000000000000000 P d1b9e01f1698456498b69ffbb2fa83bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1b9e01f1698456498b69ffbb2fa83bc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35427 } }
I20260806 09:33:59.606033 18815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:59.583285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54635 (local address 127.0.0.1:35427)
0806 09:33:59.584201 (+ 916us) server_negotiation.cc:206] Beginning negotiation
0806 09:33:59.584213 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 09:33:59.584261 (+ 48us) server_negotiation.cc:407] Connection header received
0806 09:33:59.586120 (+ 1859us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:33:59.586143 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:33:59.586534 (+ 391us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:33:59.586868 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:33:59.588918 (+ 2050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:59.590263 (+ 1345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:59.592071 (+ 1808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:59.592885 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:59.593104 (+ 219us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:59.598076 (+ 4972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:33:59.598083 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:33:59.598101 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:33:59.598141 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:33:59.601443 (+ 3302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:33:59.602220 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:33:59.602226 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:33:59.602233 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:33:59.602399 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:33:59.602811 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:33:59.602817 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:33:59.602825 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:33:59.603378 (+ 553us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:33:59.603717 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0806 09:33:59.604163 (+ 446us) server_negotiation.cc:299] Negotiation successful
0806 09:33:59.604648 (+ 485us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":702,"thread_start_us":411,"threads_started":1}
I20260806 09:33:59.606036 18811 raft_consensus.cc:388] T 00000000000000000000000000000000 P d1b9e01f1698456498b69ffbb2fa83bc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 09:33:59.606232 18811 raft_consensus.cc:482] T 00000000000000000000000000000000 P d1b9e01f1698456498b69ffbb2fa83bc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 09:33:59.606460 18811 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d1b9e01f1698456498b69ffbb2fa83bc [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:33:59.609227 18811 raft_consensus.cc:504] T 00000000000000000000000000000000 P d1b9e01f1698456498b69ffbb2fa83bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1b9e01f1698456498b69ffbb2fa83bc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35427 } }
I20260806 09:33:59.609930 20932 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 09:33:59.609781 18811 leader_election.cc:302] T 00000000000000000000000000000000 P d1b9e01f1698456498b69ffbb2fa83bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d1b9e01f1698456498b69ffbb2fa83bc; no voters:
I20260806 09:33:59.610486 18811 leader_election.cc:288] T 00000000000000000000000000000000 P d1b9e01f1698456498b69ffbb2fa83bc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 09:33:59.610638 18817 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d1b9e01f1698456498b69ffbb2fa83bc [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:33:59.612534 18811 sys_catalog.cc:564] T 00000000000000000000000000000000 P d1b9e01f1698456498b69ffbb2fa83bc [sys.catalog]: configured and running, proceeding with master startup.
I20260806 09:33:59.616338 18817 raft_consensus.cc:686] T 00000000000000000000000000000000 P d1b9e01f1698456498b69ffbb2fa83bc [term 1 LEADER]: Becoming Leader. State: Replica: d1b9e01f1698456498b69ffbb2fa83bc, State: Running, Role: LEADER
I20260806 09:33:59.616963 18817 consensus_queue.cc:237] T 00000000000000000000000000000000 P d1b9e01f1698456498b69ffbb2fa83bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1b9e01f1698456498b69ffbb2fa83bc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35427 } }
I20260806 09:33:59.620085 18819 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1b9e01f1698456498b69ffbb2fa83bc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d1b9e01f1698456498b69ffbb2fa83bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1b9e01f1698456498b69ffbb2fa83bc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35427 } } }
I20260806 09:33:59.620133 18820 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1b9e01f1698456498b69ffbb2fa83bc [sys.catalog]: SysCatalogTable state changed. Reason: New leader d1b9e01f1698456498b69ffbb2fa83bc. Latest consensus state: current_term: 1 leader_uuid: "d1b9e01f1698456498b69ffbb2fa83bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1b9e01f1698456498b69ffbb2fa83bc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35427 } } }
I20260806 09:33:59.620611 18820 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1b9e01f1698456498b69ffbb2fa83bc [sys.catalog]: This master's current role is: LEADER
I20260806 09:33:59.620608 18819 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1b9e01f1698456498b69ffbb2fa83bc [sys.catalog]: This master's current role is: LEADER
I20260806 09:33:59.621665 18832 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 09:33:59.632490 18832 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 09:33:59.640455 18815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:33:59.616594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37477 (local address 127.0.0.1:35427)
0806 09:33:59.616777 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0806 09:33:59.616784 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:33:59.617794 (+ 1010us) server_negotiation.cc:407] Connection header received
0806 09:33:59.617994 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:33:59.618002 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:33:59.618109 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:33:59.618234 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:33:59.620801 (+ 2567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:59.622256 (+ 1455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:59.624776 (+ 2520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:33:59.625747 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:33:59.625905 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:33:59.630308 (+ 4403us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:33:59.630316 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:33:59.630323 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:33:59.630386 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:33:59.635830 (+ 5444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:33:59.637050 (+ 1220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:33:59.637059 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:33:59.637065 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:33:59.637274 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:33:59.637859 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:33:59.637871 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:33:59.637877 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:33:59.638423 (+ 546us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:33:59.638730 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0806 09:33:59.639559 (+ 829us) server_negotiation.cc:299] Negotiation successful
0806 09:33:59.639889 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 09:33:59.652017 18832 catalog_manager.cc:1347] Generated new cluster ID: 006ebbee42d9445b8c3775762b2518e4
I20260806 09:33:59.652194 18832 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 09:33:59.704125 18832 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 09:33:59.705591 18832 catalog_manager.cc:1504] Loading token signing keys...
I20260806 09:33:59.716152 18832 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d1b9e01f1698456498b69ffbb2fa83bc: Generated new TSK 0
I20260806 09:33:59.717116 18832 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 09:33:59.734117 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 18696
I20260806 09:33:59.735484 18806 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:33:59.834766 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 18696
2026-08-06T09:33:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (2418 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786008495763054-20932-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 09:33:59 dist-test-slave-45dx krb5kdc[18864](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 09:33:59 dist-test-slave-45dx krb5kdc[18864](info): set up 2 sockets
Aug 06 09:33:59 dist-test-slave-45dx krb5kdc[18864](info): commencing operation
krb5kdc: starting...
W20260806 09:34:01.174988 20932 mini_kdc.cc:121] Time spent starting KDC: real 1.293s user 0.000s sys 0.063s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786008495763054-20932-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786008495763054-20932-0/krb5kdc/test-user.keytab.
Aug 06 09:34:01 dist-test-slave-45dx krb5kdc[18864](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008841, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T09:34:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T09:34:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786008495763054-20932-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786008495763054-20932-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 09:34:01.540153 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786008495763054-20932-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786008495763054-20932-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786008495763054-20932-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42757
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786008495763054-20932-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42757
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:34:01.889714 18880 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:34:01.890017 18880 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:34:01.890180 18880 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:34:01.897336 18880 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 09:34:01.897446 18880 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:34:01.897562 18880 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:34:01.897675 18880 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 09:34:01.897755 18880 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 09:34:01.907552 18880 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786008495763054-20932-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786008495763054-20932-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42757
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786008495763054-20932-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42757
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786008495763054-20932-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:34:01.908906 18880 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:34:01.910229 18880 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:34:01.922114 18901 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:34:01.922274 18898 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:34:01.922140 18895 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:34:01.923020 18880 server_base.cc:1034] running on GCE node
I20260806 09:34:01.923893 18880 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:34:01.925362 18880 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:34:01.926597 18880 hybrid_clock.cc:648] HybridClock initialized: now 1786008841926557 us; error 38 us; skew 500 ppm
I20260806 09:34:01.926981 18880 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:34:01 dist-test-slave-45dx krb5kdc[18864](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008841, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:34:01.934757 18880 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 09:34:01.936605 18880 webserver.cc:460] Webserver started at http://127.0.0.1:38539/ using document root <none> and password file <none>
I20260806 09:34:01.937388 18880 fs_manager.cc:359] Metadata directory not provided
I20260806 09:34:01.937563 18880 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:34:01.937922 18880 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:34:01.941290 18880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786008495763054-20932-0/minicluster-data/master-0/data/instance:
uuid: "69729e2051e549818897bd222d40cb49"
format_stamp: "Formatted at 2026-08-06 09:34:01 on dist-test-slave-45dx"
server_key: "45e98cbc39fa69bb6ae13a76203ddc8b"
server_key_iv: "48b6bbd94c76ae16e0cab525925b262a"
server_key_version: "encryptionkey@0"
I20260806 09:34:01.942175 18880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786008495763054-20932-0/minicluster-data/master-0/wal/instance:
uuid: "69729e2051e549818897bd222d40cb49"
format_stamp: "Formatted at 2026-08-06 09:34:01 on dist-test-slave-45dx"
server_key: "45e98cbc39fa69bb6ae13a76203ddc8b"
server_key_iv: "48b6bbd94c76ae16e0cab525925b262a"
server_key_version: "encryptionkey@0"
I20260806 09:34:01.947960 18880 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260806 09:34:01.951850 18912 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:34:01.953367 18880 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 09:34:01.953563 18880 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786008495763054-20932-0/minicluster-data/master-0/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786008495763054-20932-0/minicluster-data/master-0/wal
uuid: "69729e2051e549818897bd222d40cb49"
format_stamp: "Formatted at 2026-08-06 09:34:01 on dist-test-slave-45dx"
server_key: "45e98cbc39fa69bb6ae13a76203ddc8b"
server_key_iv: "48b6bbd94c76ae16e0cab525925b262a"
server_key_version: "encryptionkey@0"
I20260806 09:34:01.953781 18880 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786008495763054-20932-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786008495763054-20932-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786008495763054-20932-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:34:02.001475 18880 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:34:02.006003 18880 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:34:02.006348 18880 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:34:02.027314 18880 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42757
I20260806 09:34:02.027356 19007 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42757 every 8 connection(s)
I20260806 09:34:02.029021 18880 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
I20260806 09:34:02.032938 19009 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:34:02.038017 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 18880
I20260806 09:34:02.038228 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786008495763054-20932-0/minicluster-data/master-0/wal/instance
I20260806 09:34:02.038818 20932 external_mini_cluster.cc:1428] Setting key 6fc3a69613d0439140cb105c0a17f6a1
I20260806 09:34:02.044713 19009 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69729e2051e549818897bd222d40cb49: Bootstrap starting.
I20260806 09:34:02.048470 19009 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 69729e2051e549818897bd222d40cb49: Neither blocks nor log segments found. Creating new log.
I20260806 09:34:02.049810 19009 log.cc:824] T 00000000000000000000000000000000 P 69729e2051e549818897bd222d40cb49: Log is configured to *not* fsync() on all Append() calls
I20260806 09:34:02.052950 19015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:02.040787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47035 (local address 127.0.0.1:42757)
0806 09:34:02.041649 (+ 862us) server_negotiation.cc:206] Beginning negotiation
0806 09:34:02.041660 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 09:34:02.041705 (+ 45us) server_negotiation.cc:407] Connection header received
0806 09:34:02.043450 (+ 1745us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:34:02.043473 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:34:02.043853 (+ 380us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:34:02.044171 (+ 318us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:34:02.044904 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:02.046225 (+ 1321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:02.048037 (+ 1812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:02.048826 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:02.048992 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:34:02.050217 (+ 1225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:34:02.050229 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:34:02.050242 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:34:02.051089 (+ 847us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:34:02.051195 (+ 106us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:34:02.051228 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:34:02.051351 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0806 09:34:02.051640 (+ 289us) server_negotiation.cc:299] Negotiation successful
0806 09:34:02.051940 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":653,"thread_start_us":427,"threads_started":1}
I20260806 09:34:02.053424 19009 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69729e2051e549818897bd222d40cb49: No bootstrap required, opened a new log
I20260806 09:34:02.056619 20932 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 09:34:02.064764 19009 raft_consensus.cc:348] T 00000000000000000000000000000000 P 69729e2051e549818897bd222d40cb49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69729e2051e549818897bd222d40cb49" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42757 } }
I20260806 09:34:02.065316 19009 raft_consensus.cc:374] T 00000000000000000000000000000000 P 69729e2051e549818897bd222d40cb49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:34:02.065428 19009 raft_consensus.cc:729] T 00000000000000000000000000000000 P 69729e2051e549818897bd222d40cb49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69729e2051e549818897bd222d40cb49, State: Initialized, Role: FOLLOWER
I20260806 09:34:02.066037 19009 consensus_queue.cc:260] T 00000000000000000000000000000000 P 69729e2051e549818897bd222d40cb49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69729e2051e549818897bd222d40cb49" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42757 } }
I20260806 09:34:02.066308 19009 raft_consensus.cc:388] T 00000000000000000000000000000000 P 69729e2051e549818897bd222d40cb49 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 09:34:02.066501 19009 raft_consensus.cc:482] T 00000000000000000000000000000000 P 69729e2051e549818897bd222d40cb49 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 09:34:02.066722 19009 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 69729e2051e549818897bd222d40cb49 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:34:02.070006 19015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:02.062739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40209 (local address 127.0.0.1:42757)
0806 09:34:02.062966 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0806 09:34:02.062973 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:34:02.063324 (+ 351us) server_negotiation.cc:407] Connection header received
0806 09:34:02.063496 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:34:02.063502 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:34:02.063595 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:34:02.063729 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:34:02.064280 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:02.065497 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:02.067172 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:02.067921 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:02.068059 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:34:02.069213 (+ 1154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:34:02.069220 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:34:02.069225 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:34:02.069251 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:34:02.069308 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:34:02.069332 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:34:02.069436 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0806 09:34:02.069551 (+ 115us) server_negotiation.cc:299] Negotiation successful
0806 09:34:02.069646 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 09:34:02.070449 19009 raft_consensus.cc:504] T 00000000000000000000000000000000 P 69729e2051e549818897bd222d40cb49 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69729e2051e549818897bd222d40cb49" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42757 } }
I20260806 09:34:02.071000 19009 leader_election.cc:302] T 00000000000000000000000000000000 P 69729e2051e549818897bd222d40cb49 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 69729e2051e549818897bd222d40cb49; no voters:
I20260806 09:34:02.071673 19009 leader_election.cc:288] T 00000000000000000000000000000000 P 69729e2051e549818897bd222d40cb49 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 09:34:02.071719 19025 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 69729e2051e549818897bd222d40cb49 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:34:02.072508 19025 raft_consensus.cc:686] T 00000000000000000000000000000000 P 69729e2051e549818897bd222d40cb49 [term 1 LEADER]: Becoming Leader. State: Replica: 69729e2051e549818897bd222d40cb49, State: Running, Role: LEADER
I20260806 09:34:02.073118 19025 consensus_queue.cc:237] T 00000000000000000000000000000000 P 69729e2051e549818897bd222d40cb49 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69729e2051e549818897bd222d40cb49" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42757 } }
I20260806 09:34:02.073974 19009 sys_catalog.cc:564] T 00000000000000000000000000000000 P 69729e2051e549818897bd222d40cb49 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 09:34:02.076629 19026 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69729e2051e549818897bd222d40cb49 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "69729e2051e549818897bd222d40cb49" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69729e2051e549818897bd222d40cb49" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42757 } } }
I20260806 09:34:02.077147 19026 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69729e2051e549818897bd222d40cb49 [sys.catalog]: This master's current role is: LEADER
I20260806 09:34:02.078433 19039 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 09:34:02.078423 19030 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69729e2051e549818897bd222d40cb49 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 69729e2051e549818897bd222d40cb49. Latest consensus state: current_term: 1 leader_uuid: "69729e2051e549818897bd222d40cb49" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69729e2051e549818897bd222d40cb49" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42757 } } }
I20260806 09:34:02.078977 19030 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69729e2051e549818897bd222d40cb49 [sys.catalog]: This master's current role is: LEADER
I20260806 09:34:02.084205 19039 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 09:34:02.094511 19039 catalog_manager.cc:1347] Generated new cluster ID: b5a9da219a414363be2a4185754549c5
I20260806 09:34:02.094655 19039 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 09:34:02.128827 19039 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 09:34:02.130885 19039 catalog_manager.cc:1504] Loading token signing keys...
I20260806 09:34:02.154610 19039 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 69729e2051e549818897bd222d40cb49: Generated new TSK 0
I20260806 09:34:02.155591 19039 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 09:34:02.169602 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 18880
W20260806 09:34:02.170698 19002 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:47035
W20260806 09:34:02.171443 20932 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 18880: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 09:34:02.171626 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 18880
2026-08-06T09:34:02Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (2338 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786008495763054-20932-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 09:34:02 dist-test-slave-45dx krb5kdc[19064](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 09:34:02 dist-test-slave-45dx krb5kdc[19064](info): set up 2 sockets
Aug 06 09:34:02 dist-test-slave-45dx krb5kdc[19064](info): commencing operation
krb5kdc: starting...
W20260806 09:34:03.613080 20932 mini_kdc.cc:121] Time spent starting KDC: real 1.393s user 0.004s sys 0.060s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786008495763054-20932-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786008495763054-20932-0/krb5kdc/test-user.keytab.
Aug 06 09:34:03 dist-test-slave-45dx krb5kdc[19064](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008843, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T09:34:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T09:34:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786008495763054-20932-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786008495763054-20932-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 09:34:03.972769 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786008495763054-20932-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786008495763054-20932-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786008495763054-20932-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41915
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786008495763054-20932-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41915
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:34:04.322327 19084 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:34:04.322631 19084 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:34:04.322824 19084 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:34:04.330263 19084 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 09:34:04.330379 19084 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:34:04.330510 19084 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:34:04.330617 19084 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 09:34:04.330703 19084 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 09:34:04.340803 19084 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786008495763054-20932-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786008495763054-20932-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41915
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786008495763054-20932-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41915
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786008495763054-20932-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:34:04.342203 19084 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:34:04.343551 19084 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:34:04.355306 19090 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:34:04.355464 19095 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:34:04.355486 19092 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:34:04.356499 19084 server_base.cc:1034] running on GCE node
I20260806 09:34:04.357240 19084 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:34:04.358561 19084 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:34:04.359760 19084 hybrid_clock.cc:648] HybridClock initialized: now 1786008844359726 us; error 27 us; skew 500 ppm
I20260806 09:34:04.360101 19084 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:34:04 dist-test-slave-45dx krb5kdc[19064](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008844, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:34:04.363303 19084 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 09:34:04.364749 19084 webserver.cc:460] Webserver started at http://127.0.0.1:38705/ using document root <none> and password file <none>
I20260806 09:34:04.365374 19084 fs_manager.cc:359] Metadata directory not provided
I20260806 09:34:04.365518 19084 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:34:04.365820 19084 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:34:04.368664 19084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786008495763054-20932-0/minicluster-data/master-0/data/instance:
uuid: "7121af1a344c4db8b6359ddfc96f1f89"
format_stamp: "Formatted at 2026-08-06 09:34:04 on dist-test-slave-45dx"
server_key: "b39ae04a5f4af7e858885c2bcfd128a6"
server_key_iv: "0457e484a927342eb77b2df4d1b543bd"
server_key_version: "encryptionkey@0"
I20260806 09:34:04.369380 19084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786008495763054-20932-0/minicluster-data/master-0/wal/instance:
uuid: "7121af1a344c4db8b6359ddfc96f1f89"
format_stamp: "Formatted at 2026-08-06 09:34:04 on dist-test-slave-45dx"
server_key: "b39ae04a5f4af7e858885c2bcfd128a6"
server_key_iv: "0457e484a927342eb77b2df4d1b543bd"
server_key_version: "encryptionkey@0"
I20260806 09:34:04.374230 19084 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 09:34:04.377570 19108 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:34:04.378883 19084 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 09:34:04.379092 19084 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786008495763054-20932-0/minicluster-data/master-0/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786008495763054-20932-0/minicluster-data/master-0/wal
uuid: "7121af1a344c4db8b6359ddfc96f1f89"
format_stamp: "Formatted at 2026-08-06 09:34:04 on dist-test-slave-45dx"
server_key: "b39ae04a5f4af7e858885c2bcfd128a6"
server_key_iv: "0457e484a927342eb77b2df4d1b543bd"
server_key_version: "encryptionkey@0"
I20260806 09:34:04.379336 19084 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786008495763054-20932-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786008495763054-20932-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786008495763054-20932-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:34:04.385115 19084 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:34:04.389002 19084 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:34:04.389330 19084 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:34:04.407773 19084 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41915
I20260806 09:34:04.407852 19184 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41915 every 8 connection(s)
I20260806 09:34:04.409314 19084 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
I20260806 09:34:04.412983 19185 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:34:04.416777 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 19084
I20260806 09:34:04.416971 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786008495763054-20932-0/minicluster-data/master-0/wal/instance
I20260806 09:34:04.417549 20932 external_mini_cluster.cc:1428] Setting key 99b0ca607560ddc272a27601e5fb028c
I20260806 09:34:04.425360 19191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:04.419313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35659 (local address 127.0.0.1:41915)
0806 09:34:04.420169 (+ 856us) server_negotiation.cc:206] Beginning negotiation
0806 09:34:04.420180 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 09:34:04.420220 (+ 40us) server_negotiation.cc:407] Connection header received
0806 09:34:04.421908 (+ 1688us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:34:04.421929 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:34:04.422251 (+ 322us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:34:04.422571 (+ 320us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:34:04.423750 (+ 1179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:34:04.423756 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:34:04.423772 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:34:04.423791 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:34:04.423887 (+ 96us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:34:04.423909 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:34:04.424094 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0806 09:34:04.424242 (+ 148us) server_negotiation.cc:299] Negotiation successful
0806 09:34:04.424472 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":643,"thread_start_us":413,"threads_started":1}
I20260806 09:34:04.426054 19185 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7121af1a344c4db8b6359ddfc96f1f89: Bootstrap starting.
I20260806 09:34:04.429071 20932 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 09:34:04.429447 19185 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7121af1a344c4db8b6359ddfc96f1f89: Neither blocks nor log segments found. Creating new log.
I20260806 09:34:04.430447 19185 log.cc:824] T 00000000000000000000000000000000 P 7121af1a344c4db8b6359ddfc96f1f89: Log is configured to *not* fsync() on all Append() calls
I20260806 09:34:04.433564 19185 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7121af1a344c4db8b6359ddfc96f1f89: No bootstrap required, opened a new log
I20260806 09:34:04.436174 19191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:04.433403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45569 (local address 127.0.0.1:41915)
0806 09:34:04.433552 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0806 09:34:04.433556 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 09:34:04.434148 (+ 592us) server_negotiation.cc:407] Connection header received
0806 09:34:04.434346 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:34:04.434353 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:34:04.434423 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:34:04.434529 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:34:04.435490 (+ 961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:34:04.435496 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:34:04.435500 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:34:04.435518 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:34:04.435571 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:34:04.435586 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:34:04.435729 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0806 09:34:04.435829 (+ 100us) server_negotiation.cc:299] Negotiation successful
0806 09:34:04.435893 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 09:34:04.441098 19185 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7121af1a344c4db8b6359ddfc96f1f89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7121af1a344c4db8b6359ddfc96f1f89" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41915 } }
I20260806 09:34:04.441460 19185 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7121af1a344c4db8b6359ddfc96f1f89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:34:04.441569 19185 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7121af1a344c4db8b6359ddfc96f1f89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7121af1a344c4db8b6359ddfc96f1f89, State: Initialized, Role: FOLLOWER
I20260806 09:34:04.442137 19185 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7121af1a344c4db8b6359ddfc96f1f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7121af1a344c4db8b6359ddfc96f1f89" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41915 } }
I20260806 09:34:04.442384 19185 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7121af1a344c4db8b6359ddfc96f1f89 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 09:34:04.442561 19185 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7121af1a344c4db8b6359ddfc96f1f89 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 09:34:04.442754 19185 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7121af1a344c4db8b6359ddfc96f1f89 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:34:04.445065 19185 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7121af1a344c4db8b6359ddfc96f1f89 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7121af1a344c4db8b6359ddfc96f1f89" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41915 } }
I20260806 09:34:04.445616 19185 leader_election.cc:302] T 00000000000000000000000000000000 P 7121af1a344c4db8b6359ddfc96f1f89 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7121af1a344c4db8b6359ddfc96f1f89; no voters:
I20260806 09:34:04.446102 19185 leader_election.cc:288] T 00000000000000000000000000000000 P 7121af1a344c4db8b6359ddfc96f1f89 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 09:34:04.446234 19202 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7121af1a344c4db8b6359ddfc96f1f89 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:34:04.447003 19202 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7121af1a344c4db8b6359ddfc96f1f89 [term 1 LEADER]: Becoming Leader. State: Replica: 7121af1a344c4db8b6359ddfc96f1f89, State: Running, Role: LEADER
I20260806 09:34:04.447599 19202 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7121af1a344c4db8b6359ddfc96f1f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7121af1a344c4db8b6359ddfc96f1f89" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41915 } }
I20260806 09:34:04.448292 19185 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7121af1a344c4db8b6359ddfc96f1f89 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 09:34:04.450629 19206 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7121af1a344c4db8b6359ddfc96f1f89 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7121af1a344c4db8b6359ddfc96f1f89. Latest consensus state: current_term: 1 leader_uuid: "7121af1a344c4db8b6359ddfc96f1f89" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7121af1a344c4db8b6359ddfc96f1f89" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41915 } } }
I20260806 09:34:04.450642 19203 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7121af1a344c4db8b6359ddfc96f1f89 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7121af1a344c4db8b6359ddfc96f1f89" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7121af1a344c4db8b6359ddfc96f1f89" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41915 } } }
I20260806 09:34:04.451128 19206 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7121af1a344c4db8b6359ddfc96f1f89 [sys.catalog]: This master's current role is: LEADER
I20260806 09:34:04.451146 19203 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7121af1a344c4db8b6359ddfc96f1f89 [sys.catalog]: This master's current role is: LEADER
W20260806 09:34:04.463594 19227 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7121af1a344c4db8b6359ddfc96f1f89: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 09:34:04.463744 19227 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 09:34:04.463891 19213 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 09:34:04.465659 19213 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 09:34:04.474045 19213 catalog_manager.cc:1347] Generated new cluster ID: c2c801dd790442b197937bbb8ee06709
I20260806 09:34:04.474159 19213 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 09:34:04.491763 19213 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 09:34:04.492180 19213 catalog_manager.cc:1504] Loading token signing keys...
I20260806 09:34:04.502640 19213 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7121af1a344c4db8b6359ddfc96f1f89: Generated new TSK 0
I20260806 09:34:04.503643 19213 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 09:34:04.521695 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 19084
W20260806 09:34:04.523540 19179 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:35659
W20260806 09:34:04.524269 20932 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 19084: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 09:34:04.524470 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 19084
2026-08-06T09:34:04Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (2354 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786008495763054-20932-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 09:34:04 dist-test-slave-45dx krb5kdc[19237](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 09:34:04 dist-test-slave-45dx krb5kdc[19237](info): set up 2 sockets
Aug 06 09:34:04 dist-test-slave-45dx krb5kdc[19237](info): commencing operation
krb5kdc: starting...
W20260806 09:34:05.862802 20932 mini_kdc.cc:121] Time spent starting KDC: real 1.288s user 0.002s sys 0.057s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786008495763054-20932-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786008495763054-20932-0/krb5kdc/test-user.keytab.
Aug 06 09:34:06 dist-test-slave-45dx krb5kdc[19237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008846, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T09:34:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T09:34:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786008495763054-20932-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786008495763054-20932-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 09:34:06.207508 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786008495763054-20932-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786008495763054-20932-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786008495763054-20932-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34609
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786008495763054-20932-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34609
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:34:06.550515 19256 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:34:06.550802 19256 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:34:06.550899 19256 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:34:06.557767 19256 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 09:34:06.557876 19256 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:34:06.557921 19256 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:34:06.557972 19256 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 09:34:06.558008 19256 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 09:34:06.567333 19256 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786008495763054-20932-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786008495763054-20932-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34609
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786008495763054-20932-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34609
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786008495763054-20932-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:34:06.568588 19256 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:34:06.569835 19256 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:34:06.581586 19265 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:34:06.581624 19270 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:34:06.581926 19266 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:34:06.582703 19256 server_base.cc:1034] running on GCE node
I20260806 09:34:06.583477 19256 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:34:06.584892 19256 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:34:06.586126 19256 hybrid_clock.cc:648] HybridClock initialized: now 1786008846586095 us; error 42 us; skew 500 ppm
I20260806 09:34:06.586476 19256 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:34:06 dist-test-slave-45dx krb5kdc[19237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008846, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:34:06.589571 19256 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 09:34:06.591022 19256 webserver.cc:460] Webserver started at http://127.0.0.1:42711/ using document root <none> and password file <none>
I20260806 09:34:06.591665 19256 fs_manager.cc:359] Metadata directory not provided
I20260806 09:34:06.591806 19256 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:34:06.592101 19256 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:34:06.595075 19256 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786008495763054-20932-0/minicluster-data/master-0/data/instance:
uuid: "e778b248a44b44c49e8165931a14d861"
format_stamp: "Formatted at 2026-08-06 09:34:06 on dist-test-slave-45dx"
server_key: "6410c378d5c04105ff24102b48fda14e"
server_key_iv: "cb2bc255839a0799f17b4c1b7c7ffe21"
server_key_version: "encryptionkey@0"
I20260806 09:34:06.595834 19256 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786008495763054-20932-0/minicluster-data/master-0/wal/instance:
uuid: "e778b248a44b44c49e8165931a14d861"
format_stamp: "Formatted at 2026-08-06 09:34:06 on dist-test-slave-45dx"
server_key: "6410c378d5c04105ff24102b48fda14e"
server_key_iv: "cb2bc255839a0799f17b4c1b7c7ffe21"
server_key_version: "encryptionkey@0"
I20260806 09:34:06.601076 19256 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260806 09:34:06.604552 19278 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:34:06.605811 19256 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 09:34:06.606015 19256 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786008495763054-20932-0/minicluster-data/master-0/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786008495763054-20932-0/minicluster-data/master-0/wal
uuid: "e778b248a44b44c49e8165931a14d861"
format_stamp: "Formatted at 2026-08-06 09:34:06 on dist-test-slave-45dx"
server_key: "6410c378d5c04105ff24102b48fda14e"
server_key_iv: "cb2bc255839a0799f17b4c1b7c7ffe21"
server_key_version: "encryptionkey@0"
I20260806 09:34:06.606206 19256 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786008495763054-20932-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786008495763054-20932-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786008495763054-20932-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:34:06.617707 19256 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:34:06.621481 19256 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:34:06.621775 19256 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:34:06.639866 19256 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34609
I20260806 09:34:06.639901 19365 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34609 every 8 connection(s)
I20260806 09:34:06.641271 19256 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
I20260806 09:34:06.645046 19366 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:34:06.651311 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 19256
I20260806 09:34:06.651515 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786008495763054-20932-0/minicluster-data/master-0/wal/instance
I20260806 09:34:06.652057 20932 external_mini_cluster.cc:1428] Setting key 4e3ae952ffea6b2fd50e3a0162d78b64
I20260806 09:34:06.655543 19366 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e778b248a44b44c49e8165931a14d861: Bootstrap starting.
I20260806 09:34:06.659149 19366 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e778b248a44b44c49e8165931a14d861: Neither blocks nor log segments found. Creating new log.
I20260806 09:34:06.660373 19366 log.cc:824] T 00000000000000000000000000000000 P e778b248a44b44c49e8165931a14d861: Log is configured to *not* fsync() on all Append() calls
I20260806 09:34:06.663340 19366 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e778b248a44b44c49e8165931a14d861: No bootstrap required, opened a new log
Aug 06 09:34:06 dist-test-slave-45dx krb5kdc[19237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008846, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 09:34:06.671555 19366 raft_consensus.cc:348] T 00000000000000000000000000000000 P e778b248a44b44c49e8165931a14d861 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e778b248a44b44c49e8165931a14d861" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34609 } }
I20260806 09:34:06.671949 19366 raft_consensus.cc:374] T 00000000000000000000000000000000 P e778b248a44b44c49e8165931a14d861 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:34:06.672062 19366 raft_consensus.cc:729] T 00000000000000000000000000000000 P e778b248a44b44c49e8165931a14d861 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e778b248a44b44c49e8165931a14d861, State: Initialized, Role: FOLLOWER
I20260806 09:34:06.672739 19366 consensus_queue.cc:260] T 00000000000000000000000000000000 P e778b248a44b44c49e8165931a14d861 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e778b248a44b44c49e8165931a14d861" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34609 } }
I20260806 09:34:06.673051 19366 raft_consensus.cc:388] T 00000000000000000000000000000000 P e778b248a44b44c49e8165931a14d861 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 09:34:06.673197 19366 raft_consensus.cc:482] T 00000000000000000000000000000000 P e778b248a44b44c49e8165931a14d861 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 09:34:06.673409 19366 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e778b248a44b44c49e8165931a14d861 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:34:06.675858 19369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:06.653967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.37:43817 (local address 127.0.0.1:34609)
0806 09:34:06.654730 (+ 763us) server_negotiation.cc:206] Beginning negotiation
0806 09:34:06.654741 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 09:34:06.654780 (+ 39us) server_negotiation.cc:407] Connection header received
0806 09:34:06.656524 (+ 1744us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:34:06.656547 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:34:06.656913 (+ 366us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:34:06.657224 (+ 311us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:34:06.659028 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:06.660599 (+ 1571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:06.662210 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:06.662832 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:06.662960 (+ 128us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:34:06.667646 (+ 4686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:34:06.667653 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:34:06.667666 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:34:06.667700 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:34:06.670963 (+ 3263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:34:06.671707 (+ 744us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:34:06.671714 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:34:06.671720 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:34:06.671910 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:34:06.672523 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:34:06.672528 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:34:06.672531 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:34:06.673284 (+ 753us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:34:06.673651 (+ 367us) server_negotiation.cc:1035] Waiting for connection context
0806 09:34:06.674174 (+ 523us) server_negotiation.cc:299] Negotiation successful
0806 09:34:06.674531 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":575,"thread_start_us":381,"threads_started":1}
I20260806 09:34:06.676676 19366 raft_consensus.cc:504] T 00000000000000000000000000000000 P e778b248a44b44c49e8165931a14d861 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e778b248a44b44c49e8165931a14d861" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34609 } }
I20260806 09:34:06.677161 19366 leader_election.cc:302] T 00000000000000000000000000000000 P e778b248a44b44c49e8165931a14d861 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e778b248a44b44c49e8165931a14d861; no voters:
I20260806 09:34:06.677819 19366 leader_election.cc:288] T 00000000000000000000000000000000 P e778b248a44b44c49e8165931a14d861 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 09:34:06.677946 19376 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e778b248a44b44c49e8165931a14d861 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:34:06.679221 19376 raft_consensus.cc:686] T 00000000000000000000000000000000 P e778b248a44b44c49e8165931a14d861 [term 1 LEADER]: Becoming Leader. State: Replica: e778b248a44b44c49e8165931a14d861, State: Running, Role: LEADER
I20260806 09:34:06.680008 19366 sys_catalog.cc:564] T 00000000000000000000000000000000 P e778b248a44b44c49e8165931a14d861 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 09:34:06.679874 19376 consensus_queue.cc:237] T 00000000000000000000000000000000 P e778b248a44b44c49e8165931a14d861 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e778b248a44b44c49e8165931a14d861" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34609 } }
I20260806 09:34:06.680519 20932 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 09:34:06.686057 19377 sys_catalog.cc:455] T 00000000000000000000000000000000 P e778b248a44b44c49e8165931a14d861 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e778b248a44b44c49e8165931a14d861" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e778b248a44b44c49e8165931a14d861" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34609 } } }
I20260806 09:34:06.686501 19377 sys_catalog.cc:458] T 00000000000000000000000000000000 P e778b248a44b44c49e8165931a14d861 [sys.catalog]: This master's current role is: LEADER
I20260806 09:34:06.687235 19378 sys_catalog.cc:455] T 00000000000000000000000000000000 P e778b248a44b44c49e8165931a14d861 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e778b248a44b44c49e8165931a14d861. Latest consensus state: current_term: 1 leader_uuid: "e778b248a44b44c49e8165931a14d861" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e778b248a44b44c49e8165931a14d861" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34609 } } }
I20260806 09:34:06.689270 19378 sys_catalog.cc:458] T 00000000000000000000000000000000 P e778b248a44b44c49e8165931a14d861 [sys.catalog]: This master's current role is: LEADER
I20260806 09:34:06.706147 19385 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 09:34:06.713774 19385 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 09:34:06.737989 19385 catalog_manager.cc:1347] Generated new cluster ID: 45b6a257c75c4ae59dc16ffc5b6c2828
I20260806 09:34:06.738148 19385 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 09:34:06.740784 19369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:06.715953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.37:57909 (local address 127.0.0.1:34609)
0806 09:34:06.716121 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0806 09:34:06.716127 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:34:06.720159 (+ 4032us) server_negotiation.cc:407] Connection header received
0806 09:34:06.720363 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:34:06.720369 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:34:06.720459 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:34:06.720592 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:34:06.722660 (+ 2068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:06.723831 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:06.725726 (+ 1895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:06.726446 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:06.726545 (+ 99us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:34:06.730374 (+ 3829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:34:06.730383 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:34:06.730388 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:34:06.730426 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:34:06.734250 (+ 3824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:34:06.735312 (+ 1062us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:34:06.735320 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:34:06.735323 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:34:06.735481 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:34:06.736124 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:34:06.736129 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:34:06.736133 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:34:06.736616 (+ 483us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:34:06.736910 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0806 09:34:06.737644 (+ 734us) server_negotiation.cc:299] Negotiation successful
0806 09:34:06.737837 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 09:34:06.761183 19385 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 09:34:06.762485 19385 catalog_manager.cc:1504] Loading token signing keys...
I20260806 09:34:06.770816 19385 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e778b248a44b44c49e8165931a14d861: Generated new TSK 0
I20260806 09:34:06.771590 19385 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 09:34:06.778626 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 19256
I20260806 09:34:06.779733 19357 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:34:06.870253 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 19256
W20260806 09:34:06.896973 19234 connection.cc:535] client connection to 127.0.0.1:34609 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34609): Connection reset by peer (error 104)
2026-08-06T09:34:06Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (2341 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786008495763054-20932-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 09:34:07 dist-test-slave-45dx krb5kdc[19419](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 09:34:07 dist-test-slave-45dx krb5kdc[19419](info): set up 2 sockets
Aug 06 09:34:07 dist-test-slave-45dx krb5kdc[19419](info): commencing operation
krb5kdc: starting...
W20260806 09:34:08.213292 20932 mini_kdc.cc:121] Time spent starting KDC: real 1.297s user 0.000s sys 0.059s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786008495763054-20932-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786008495763054-20932-0/krb5kdc/test-user.keytab.
Aug 06 09:34:08 dist-test-slave-45dx krb5kdc[19419](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008848, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T09:34:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T09:34:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786008495763054-20932-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786008495763054-20932-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 09:34:08.577982 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786008495763054-20932-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786008495763054-20932-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786008495763054-20932-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33579
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786008495763054-20932-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33579
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:34:08.926679 19435 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:34:08.927055 19435 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:34:08.927240 19435 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:34:08.934367 19435 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 09:34:08.934528 19435 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:34:08.934640 19435 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:34:08.934756 19435 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 09:34:08.934859 19435 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 09:34:08.944585 19435 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786008495763054-20932-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786008495763054-20932-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33579
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786008495763054-20932-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33579
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786008495763054-20932-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:34:08.945984 19435 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:34:08.947288 19435 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:34:08.959184 19444 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:34:08.959297 19443 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:34:08.959327 19447 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:34:08.960180 19435 server_base.cc:1034] running on GCE node
I20260806 09:34:08.961062 19435 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:34:08.962450 19435 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:34:08.963649 19435 hybrid_clock.cc:648] HybridClock initialized: now 1786008848963589 us; error 56 us; skew 500 ppm
I20260806 09:34:08.963999 19435 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:34:08 dist-test-slave-45dx krb5kdc[19419](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008848, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:34:08.967156 19435 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 09:34:08.968683 19435 webserver.cc:460] Webserver started at http://127.0.0.1:36485/ using document root <none> and password file <none>
I20260806 09:34:08.969322 19435 fs_manager.cc:359] Metadata directory not provided
I20260806 09:34:08.969452 19435 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:34:08.969807 19435 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:34:08.972942 19435 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786008495763054-20932-0/minicluster-data/master-0/data/instance:
uuid: "dafa45d88057452cb151c184e6714a36"
format_stamp: "Formatted at 2026-08-06 09:34:08 on dist-test-slave-45dx"
server_key: "f1484052f4f1e01cf1dfdd64d1dc0603"
server_key_iv: "91efe685e54cebc2139cbad4cfec6a9a"
server_key_version: "encryptionkey@0"
I20260806 09:34:08.973757 19435 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786008495763054-20932-0/minicluster-data/master-0/wal/instance:
uuid: "dafa45d88057452cb151c184e6714a36"
format_stamp: "Formatted at 2026-08-06 09:34:08 on dist-test-slave-45dx"
server_key: "f1484052f4f1e01cf1dfdd64d1dc0603"
server_key_iv: "91efe685e54cebc2139cbad4cfec6a9a"
server_key_version: "encryptionkey@0"
I20260806 09:34:08.979377 19435 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 09:34:08.983115 19456 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:34:08.984494 19435 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260806 09:34:08.984684 19435 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786008495763054-20932-0/minicluster-data/master-0/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786008495763054-20932-0/minicluster-data/master-0/wal
uuid: "dafa45d88057452cb151c184e6714a36"
format_stamp: "Formatted at 2026-08-06 09:34:08 on dist-test-slave-45dx"
server_key: "f1484052f4f1e01cf1dfdd64d1dc0603"
server_key_iv: "91efe685e54cebc2139cbad4cfec6a9a"
server_key_version: "encryptionkey@0"
I20260806 09:34:08.984886 19435 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786008495763054-20932-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786008495763054-20932-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786008495763054-20932-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:34:09.001181 19435 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:34:09.005009 19435 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:34:09.005335 19435 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:34:09.023777 19435 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33579
I20260806 09:34:09.023861 19533 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33579 every 8 connection(s)
I20260806 09:34:09.025125 19435 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
I20260806 09:34:09.028702 19534 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:34:09.033874 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 19435
I20260806 09:34:09.034072 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786008495763054-20932-0/minicluster-data/master-0/wal/instance
I20260806 09:34:09.034619 20932 external_mini_cluster.cc:1428] Setting key db626a78dedbca36dbf5f74efbf62c29
I20260806 09:34:09.039348 19534 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dafa45d88057452cb151c184e6714a36: Bootstrap starting.
I20260806 09:34:09.042896 19534 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dafa45d88057452cb151c184e6714a36: Neither blocks nor log segments found. Creating new log.
I20260806 09:34:09.044013 19534 log.cc:824] T 00000000000000000000000000000000 P dafa45d88057452cb151c184e6714a36: Log is configured to *not* fsync() on all Append() calls
I20260806 09:34:09.047204 19534 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dafa45d88057452cb151c184e6714a36: No bootstrap required, opened a new log
I20260806 09:34:09.048256 19543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:09.036468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.37:39781 (local address 127.0.0.1:33579)
0806 09:34:09.037254 (+ 786us) server_negotiation.cc:206] Beginning negotiation
0806 09:34:09.037264 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 09:34:09.037304 (+ 40us) server_negotiation.cc:407] Connection header received
0806 09:34:09.038915 (+ 1611us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:34:09.038937 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:34:09.039296 (+ 359us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:34:09.039671 (+ 375us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:34:09.040283 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:09.041635 (+ 1352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:09.043511 (+ 1876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:09.044130 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:09.044355 (+ 225us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:34:09.045673 (+ 1318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:34:09.045683 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:34:09.045699 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:34:09.046369 (+ 670us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:34:09.046473 (+ 104us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:34:09.046503 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:34:09.046689 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0806 09:34:09.046869 (+ 180us) server_negotiation.cc:299] Negotiation successful
0806 09:34:09.047049 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":587,"thread_start_us":365,"threads_started":1}
I20260806 09:34:09.052047 20932 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 09:34:09.055610 19534 raft_consensus.cc:348] T 00000000000000000000000000000000 P dafa45d88057452cb151c184e6714a36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dafa45d88057452cb151c184e6714a36" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33579 } }
I20260806 09:34:09.055979 19534 raft_consensus.cc:374] T 00000000000000000000000000000000 P dafa45d88057452cb151c184e6714a36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:34:09.056110 19534 raft_consensus.cc:729] T 00000000000000000000000000000000 P dafa45d88057452cb151c184e6714a36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dafa45d88057452cb151c184e6714a36, State: Initialized, Role: FOLLOWER
I20260806 09:34:09.056803 19534 consensus_queue.cc:260] T 00000000000000000000000000000000 P dafa45d88057452cb151c184e6714a36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dafa45d88057452cb151c184e6714a36" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33579 } }
I20260806 09:34:09.057102 19534 raft_consensus.cc:388] T 00000000000000000000000000000000 P dafa45d88057452cb151c184e6714a36 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 09:34:09.057251 19534 raft_consensus.cc:482] T 00000000000000000000000000000000 P dafa45d88057452cb151c184e6714a36 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 09:34:09.057421 19534 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dafa45d88057452cb151c184e6714a36 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:34:09.060061 19534 raft_consensus.cc:504] T 00000000000000000000000000000000 P dafa45d88057452cb151c184e6714a36 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dafa45d88057452cb151c184e6714a36" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33579 } }
I20260806 09:34:09.060702 19534 leader_election.cc:302] T 00000000000000000000000000000000 P dafa45d88057452cb151c184e6714a36 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dafa45d88057452cb151c184e6714a36; no voters:
I20260806 09:34:09.061358 19534 leader_election.cc:288] T 00000000000000000000000000000000 P dafa45d88057452cb151c184e6714a36 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 09:34:09.061499 19552 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dafa45d88057452cb151c184e6714a36 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:34:09.063383 19543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:09.056170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.37:40095 (local address 127.0.0.1:33579)
0806 09:34:09.056392 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0806 09:34:09.056397 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:34:09.056781 (+ 384us) server_negotiation.cc:407] Connection header received
0806 09:34:09.056925 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:34:09.056930 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:34:09.057007 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:34:09.057125 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:34:09.057739 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:09.058891 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:09.060156 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:09.060867 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:09.061006 (+ 139us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:34:09.062337 (+ 1331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:34:09.062344 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:34:09.062348 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:34:09.062370 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:34:09.062421 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:34:09.062446 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:34:09.062563 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0806 09:34:09.063009 (+ 446us) server_negotiation.cc:299] Negotiation successful
0806 09:34:09.063066 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 09:34:09.064029 19534 sys_catalog.cc:564] T 00000000000000000000000000000000 P dafa45d88057452cb151c184e6714a36 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 09:34:09.063987 19552 raft_consensus.cc:686] T 00000000000000000000000000000000 P dafa45d88057452cb151c184e6714a36 [term 1 LEADER]: Becoming Leader. State: Replica: dafa45d88057452cb151c184e6714a36, State: Running, Role: LEADER
I20260806 09:34:09.064589 19552 consensus_queue.cc:237] T 00000000000000000000000000000000 P dafa45d88057452cb151c184e6714a36 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dafa45d88057452cb151c184e6714a36" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33579 } }
I20260806 09:34:09.068010 19555 sys_catalog.cc:455] T 00000000000000000000000000000000 P dafa45d88057452cb151c184e6714a36 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dafa45d88057452cb151c184e6714a36. Latest consensus state: current_term: 1 leader_uuid: "dafa45d88057452cb151c184e6714a36" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dafa45d88057452cb151c184e6714a36" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33579 } } }
I20260806 09:34:09.068403 19555 sys_catalog.cc:458] T 00000000000000000000000000000000 P dafa45d88057452cb151c184e6714a36 [sys.catalog]: This master's current role is: LEADER
I20260806 09:34:09.069016 19553 sys_catalog.cc:455] T 00000000000000000000000000000000 P dafa45d88057452cb151c184e6714a36 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dafa45d88057452cb151c184e6714a36" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dafa45d88057452cb151c184e6714a36" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33579 } } }
I20260806 09:34:09.069382 19553 sys_catalog.cc:458] T 00000000000000000000000000000000 P dafa45d88057452cb151c184e6714a36 [sys.catalog]: This master's current role is: LEADER
I20260806 09:34:09.071681 19562 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 09:34:09.075812 19562 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 09:34:09.084100 19562 catalog_manager.cc:1347] Generated new cluster ID: 29398b3642a44552ae307ef8d95e638f
I20260806 09:34:09.084208 19562 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 09:34:09.096472 19562 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 09:34:09.098145 19562 catalog_manager.cc:1504] Loading token signing keys...
I20260806 09:34:09.110669 19562 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dafa45d88057452cb151c184e6714a36: Generated new TSK 0
I20260806 09:34:09.111626 19562 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 09:34:09.123672 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 19435
W20260806 09:34:09.124940 19527 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.37:39781
W20260806 09:34:09.125795 20932 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 19435: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 09:34:09.125989 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 19435
W20260806 09:34:09.149142 19416 connection.cc:535] client connection to 127.0.0.1:33579 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33579): Connection reset by peer (error 104)
2026-08-06T09:34:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (2252 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786008495763054-20932-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 09:34:09 dist-test-slave-45dx krb5kdc[19587](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 09:34:09 dist-test-slave-45dx krb5kdc[19587](info): set up 2 sockets
Aug 06 09:34:09 dist-test-slave-45dx krb5kdc[19587](info): commencing operation
krb5kdc: starting...
W20260806 09:34:10.422313 20932 mini_kdc.cc:121] Time spent starting KDC: real 1.254s user 0.000s sys 0.055s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786008495763054-20932-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786008495763054-20932-0/krb5kdc/test-user.keytab.
Aug 06 09:34:10 dist-test-slave-45dx krb5kdc[19587](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008850, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T09:34:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T09:34:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786008495763054-20932-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786008495763054-20932-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 09:34:10.781728 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786008495763054-20932-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786008495763054-20932-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786008495763054-20932-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46171
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786008495763054-20932-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37827
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46171
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:34:11.131476 19609 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:34:11.131749 19609 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:34:11.131850 19609 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:34:11.139081 19609 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 09:34:11.139199 19609 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:34:11.139246 19609 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:34:11.139308 19609 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 09:34:11.139345 19609 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 09:34:11.149171 19609 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37827
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786008495763054-20932-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786008495763054-20932-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46171
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786008495763054-20932-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46171
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786008495763054-20932-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:34:11.150377 19609 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:34:11.151659 19609 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:34:11.163538 19615 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:34:11.163499 19616 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:34:11.163726 19619 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:34:11.164470 19609 server_base.cc:1034] running on GCE node
I20260806 09:34:11.165169 19609 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:34:11.166739 19609 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:34:11.167940 19609 hybrid_clock.cc:648] HybridClock initialized: now 1786008851167819 us; error 85 us; skew 500 ppm
I20260806 09:34:11.168417 19609 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:34:11 dist-test-slave-45dx krb5kdc[19587](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008851, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:34:11.171852 19609 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 09:34:11.173290 19609 webserver.cc:460] Webserver started at http://127.0.0.1:41439/ using document root <none> and password file <none>
I20260806 09:34:11.173942 19609 fs_manager.cc:359] Metadata directory not provided
I20260806 09:34:11.174088 19609 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:34:11.174400 19609 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:34:11.177417 19609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786008495763054-20932-0/minicluster-data/master-0/data/instance:
uuid: "90835733221f45aab8ad607849fb1dbf"
format_stamp: "Formatted at 2026-08-06 09:34:11 on dist-test-slave-45dx"
server_key: "0da424548b6ddd72517b91a233cf486a"
server_key_iv: "e4efb52cc828338feeae8e3cb03ac466"
server_key_version: "encryptionkey@0"
I20260806 09:34:11.178220 19609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786008495763054-20932-0/minicluster-data/master-0/wal/instance:
uuid: "90835733221f45aab8ad607849fb1dbf"
format_stamp: "Formatted at 2026-08-06 09:34:11 on dist-test-slave-45dx"
server_key: "0da424548b6ddd72517b91a233cf486a"
server_key_iv: "e4efb52cc828338feeae8e3cb03ac466"
server_key_version: "encryptionkey@0"
I20260806 09:34:11.183285 19609 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260806 09:34:11.186594 19628 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:34:11.187861 19609 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260806 09:34:11.188042 19609 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786008495763054-20932-0/minicluster-data/master-0/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786008495763054-20932-0/minicluster-data/master-0/wal
uuid: "90835733221f45aab8ad607849fb1dbf"
format_stamp: "Formatted at 2026-08-06 09:34:11 on dist-test-slave-45dx"
server_key: "0da424548b6ddd72517b91a233cf486a"
server_key_iv: "e4efb52cc828338feeae8e3cb03ac466"
server_key_version: "encryptionkey@0"
I20260806 09:34:11.188316 19609 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786008495763054-20932-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786008495763054-20932-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786008495763054-20932-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:34:11.193711 19609 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:34:11.197937 19609 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:34:11.198217 19609 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:34:11.216130 19609 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46171
I20260806 09:34:11.216215 19704 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46171 every 8 connection(s)
I20260806 09:34:11.217561 19609 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
I20260806 09:34:11.221405 19706 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:34:11.225824 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 19609
I20260806 09:34:11.226023 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786008495763054-20932-0/minicluster-data/master-0/wal/instance
I20260806 09:34:11.226658 20932 external_mini_cluster.cc:1428] Setting key 278e0e7ea147f7587b51bb8819e56240
I20260806 09:34:11.234582 19709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:11.228654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.37:38141 (local address 127.0.0.1:46171)
0806 09:34:11.229452 (+ 798us) server_negotiation.cc:206] Beginning negotiation
0806 09:34:11.229463 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 09:34:11.229505 (+ 42us) server_negotiation.cc:407] Connection header received
0806 09:34:11.231115 (+ 1610us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:34:11.231136 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:34:11.231509 (+ 373us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:34:11.231828 (+ 319us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:34:11.232991 (+ 1163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:34:11.232997 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:34:11.233011 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:34:11.233024 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:34:11.233080 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:34:11.233099 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:34:11.233262 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0806 09:34:11.233410 (+ 148us) server_negotiation.cc:299] Negotiation successful
0806 09:34:11.233611 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":601,"thread_start_us":356,"threads_started":1}
I20260806 09:34:11.235459 19706 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90835733221f45aab8ad607849fb1dbf: Bootstrap starting.
I20260806 09:34:11.238199 20932 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 09:34:11.238909 19706 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 90835733221f45aab8ad607849fb1dbf: Neither blocks nor log segments found. Creating new log.
I20260806 09:34:11.240144 19706 log.cc:824] T 00000000000000000000000000000000 P 90835733221f45aab8ad607849fb1dbf: Log is configured to *not* fsync() on all Append() calls
I20260806 09:34:11.243505 19706 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90835733221f45aab8ad607849fb1dbf: No bootstrap required, opened a new log
I20260806 09:34:11.246142 19709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:11.243465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.37:44887 (local address 127.0.0.1:46171)
0806 09:34:11.243667 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0806 09:34:11.243673 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:34:11.244091 (+ 418us) server_negotiation.cc:407] Connection header received
0806 09:34:11.244231 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:34:11.244236 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:34:11.244330 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:34:11.244443 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:34:11.245393 (+ 950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:34:11.245403 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:34:11.245408 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:34:11.245428 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:34:11.245485 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:34:11.245504 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:34:11.245640 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0806 09:34:11.245750 (+ 110us) server_negotiation.cc:299] Negotiation successful
0806 09:34:11.245815 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260806 09:34:11.253429 19706 raft_consensus.cc:348] T 00000000000000000000000000000000 P 90835733221f45aab8ad607849fb1dbf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90835733221f45aab8ad607849fb1dbf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46171 } }
I20260806 09:34:11.254365 19706 raft_consensus.cc:374] T 00000000000000000000000000000000 P 90835733221f45aab8ad607849fb1dbf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:34:11.254568 19706 raft_consensus.cc:729] T 00000000000000000000000000000000 P 90835733221f45aab8ad607849fb1dbf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90835733221f45aab8ad607849fb1dbf, State: Initialized, Role: FOLLOWER
I20260806 09:34:11.255417 19706 consensus_queue.cc:260] T 00000000000000000000000000000000 P 90835733221f45aab8ad607849fb1dbf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90835733221f45aab8ad607849fb1dbf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46171 } }
I20260806 09:34:11.255721 19706 raft_consensus.cc:388] T 00000000000000000000000000000000 P 90835733221f45aab8ad607849fb1dbf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 09:34:11.255897 19706 raft_consensus.cc:482] T 00000000000000000000000000000000 P 90835733221f45aab8ad607849fb1dbf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 09:34:11.256121 19706 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 90835733221f45aab8ad607849fb1dbf [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:34:11.258927 19706 raft_consensus.cc:504] T 00000000000000000000000000000000 P 90835733221f45aab8ad607849fb1dbf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90835733221f45aab8ad607849fb1dbf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46171 } }
I20260806 09:34:11.259632 19706 leader_election.cc:302] T 00000000000000000000000000000000 P 90835733221f45aab8ad607849fb1dbf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 90835733221f45aab8ad607849fb1dbf; no voters:
I20260806 09:34:11.260589 19706 leader_election.cc:288] T 00000000000000000000000000000000 P 90835733221f45aab8ad607849fb1dbf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 09:34:11.260788 19717 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 90835733221f45aab8ad607849fb1dbf [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:34:11.262259 19717 raft_consensus.cc:686] T 00000000000000000000000000000000 P 90835733221f45aab8ad607849fb1dbf [term 1 LEADER]: Becoming Leader. State: Replica: 90835733221f45aab8ad607849fb1dbf, State: Running, Role: LEADER
I20260806 09:34:11.262893 19717 consensus_queue.cc:237] T 00000000000000000000000000000000 P 90835733221f45aab8ad607849fb1dbf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90835733221f45aab8ad607849fb1dbf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46171 } }
I20260806 09:34:11.263386 19706 sys_catalog.cc:564] T 00000000000000000000000000000000 P 90835733221f45aab8ad607849fb1dbf [sys.catalog]: configured and running, proceeding with master startup.
I20260806 09:34:11.266242 19723 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90835733221f45aab8ad607849fb1dbf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 90835733221f45aab8ad607849fb1dbf. Latest consensus state: current_term: 1 leader_uuid: "90835733221f45aab8ad607849fb1dbf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90835733221f45aab8ad607849fb1dbf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46171 } } }
I20260806 09:34:11.266786 19723 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90835733221f45aab8ad607849fb1dbf [sys.catalog]: This master's current role is: LEADER
I20260806 09:34:11.266675 19722 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90835733221f45aab8ad607849fb1dbf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "90835733221f45aab8ad607849fb1dbf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90835733221f45aab8ad607849fb1dbf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46171 } } }
I20260806 09:34:11.267385 19722 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90835733221f45aab8ad607849fb1dbf [sys.catalog]: This master's current role is: LEADER
I20260806 09:34:11.268549 19729 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 09:34:11.274209 19729 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 09:34:11.287020 19729 catalog_manager.cc:1347] Generated new cluster ID: 404d43a0e44f4864b7f475db67d5c93e
I20260806 09:34:11.287168 19729 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 09:34:11.313431 19729 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 09:34:11.314020 19729 catalog_manager.cc:1504] Loading token signing keys...
I20260806 09:34:11.321763 19729 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 90835733221f45aab8ad607849fb1dbf: Generated new TSK 0
I20260806 09:34:11.322757 19729 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 09:34:11.334206 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 19609
W20260806 09:34:11.335500 19698 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.37:38141
W20260806 09:34:11.336344 20932 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 19609: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 09:34:11.336558 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 19609
2026-08-06T09:34:11Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (2216 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786008495763054-20932-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 09:34:11 dist-test-slave-45dx krb5kdc[19749](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 09:34:11 dist-test-slave-45dx krb5kdc[19749](info): set up 2 sockets
Aug 06 09:34:11 dist-test-slave-45dx krb5kdc[19749](info): commencing operation
krb5kdc: starting...
W20260806 09:34:12.675082 20932 mini_kdc.cc:121] Time spent starting KDC: real 1.290s user 0.000s sys 0.062s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786008495763054-20932-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786008495763054-20932-0/krb5kdc/test-user.keytab.
Aug 06 09:34:12 dist-test-slave-45dx krb5kdc[19749](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008852, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T09:34:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T09:34:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786008495763054-20932-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786008495763054-20932-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 09:34:13.051580 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786008495763054-20932-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786008495763054-20932-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786008495763054-20932-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37579
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786008495763054-20932-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37579
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:34:13.389722 19767 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:34:13.390015 19767 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:34:13.390239 19767 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:34:13.397437 19767 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 09:34:13.397573 19767 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:34:13.397678 19767 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:34:13.397791 19767 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 09:34:13.397871 19767 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 09:34:13.407709 19767 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786008495763054-20932-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786008495763054-20932-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37579
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786008495763054-20932-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37579
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786008495763054-20932-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:34:13.409097 19767 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:34:13.410396 19767 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:34:13.422605 19784 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:34:13.422770 19782 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:34:13.422737 19781 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:34:13.424435 19767 server_base.cc:1034] running on GCE node
I20260806 09:34:13.425285 19767 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:34:13.426786 19767 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:34:13.428010 19767 hybrid_clock.cc:648] HybridClock initialized: now 1786008853427958 us; error 47 us; skew 500 ppm
I20260806 09:34:13.428432 19767 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:34:13 dist-test-slave-45dx krb5kdc[19749](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008853, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:34:13.431738 19767 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 09:34:13.433373 19767 webserver.cc:460] Webserver started at http://127.0.0.1:34439/ using document root <none> and password file <none>
I20260806 09:34:13.434075 19767 fs_manager.cc:359] Metadata directory not provided
I20260806 09:34:13.434212 19767 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:34:13.434531 19767 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:34:13.437588 19767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786008495763054-20932-0/minicluster-data/master-0/data/instance:
uuid: "46848e56a85f4383a7c1bf06569fd19b"
format_stamp: "Formatted at 2026-08-06 09:34:13 on dist-test-slave-45dx"
server_key: "e13c89d0e18adc4aa843491c145b987b"
server_key_iv: "795913ebc4516727a226eaf5c585c8f5"
server_key_version: "encryptionkey@0"
I20260806 09:34:13.438385 19767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786008495763054-20932-0/minicluster-data/master-0/wal/instance:
uuid: "46848e56a85f4383a7c1bf06569fd19b"
format_stamp: "Formatted at 2026-08-06 09:34:13 on dist-test-slave-45dx"
server_key: "e13c89d0e18adc4aa843491c145b987b"
server_key_iv: "795913ebc4516727a226eaf5c585c8f5"
server_key_version: "encryptionkey@0"
I20260806 09:34:13.443660 19767 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260806 09:34:13.447443 19794 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:34:13.448846 19767 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 09:34:13.449055 19767 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786008495763054-20932-0/minicluster-data/master-0/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786008495763054-20932-0/minicluster-data/master-0/wal
uuid: "46848e56a85f4383a7c1bf06569fd19b"
format_stamp: "Formatted at 2026-08-06 09:34:13 on dist-test-slave-45dx"
server_key: "e13c89d0e18adc4aa843491c145b987b"
server_key_iv: "795913ebc4516727a226eaf5c585c8f5"
server_key_version: "encryptionkey@0"
I20260806 09:34:13.449234 19767 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786008495763054-20932-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786008495763054-20932-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786008495763054-20932-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:34:13.477387 19767 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:34:13.482060 19767 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:34:13.482371 19767 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:34:13.500949 19767 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37579
I20260806 09:34:13.501036 19878 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37579 every 8 connection(s)
I20260806 09:34:13.502316 19767 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
I20260806 09:34:13.504191 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 19767
I20260806 09:34:13.504407 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786008495763054-20932-0/minicluster-data/master-0/wal/instance
I20260806 09:34:13.504843 20932 external_mini_cluster.cc:1428] Setting key cb16a3facba0f660826963363e71b251
I20260806 09:34:13.506501 19879 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 06 09:34:13 dist-test-slave-45dx krb5kdc[19749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008852, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 09:34:13.520380 19879 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46848e56a85f4383a7c1bf06569fd19b: Bootstrap starting.
I20260806 09:34:13.523586 19879 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 46848e56a85f4383a7c1bf06569fd19b: Neither blocks nor log segments found. Creating new log.
I20260806 09:34:13.524711 19879 log.cc:824] T 00000000000000000000000000000000 P 46848e56a85f4383a7c1bf06569fd19b: Log is configured to *not* fsync() on all Append() calls
I20260806 09:34:13.527581 19879 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46848e56a85f4383a7c1bf06569fd19b: No bootstrap required, opened a new log
I20260806 09:34:13.528395 19882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:13.506670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.37:45547 (local address 127.0.0.1:37579)
0806 09:34:13.507465 (+ 795us) server_negotiation.cc:206] Beginning negotiation
0806 09:34:13.507476 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 09:34:13.507515 (+ 39us) server_negotiation.cc:407] Connection header received
0806 09:34:13.509084 (+ 1569us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:34:13.509104 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:34:13.509482 (+ 378us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:34:13.509796 (+ 314us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:34:13.511704 (+ 1908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:13.513220 (+ 1516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:13.515033 (+ 1813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:13.515675 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:13.515806 (+ 131us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:34:13.520745 (+ 4939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:34:13.520752 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:34:13.520765 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:34:13.520803 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:34:13.524035 (+ 3232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:34:13.524921 (+ 886us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:34:13.524928 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:34:13.524934 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:34:13.525091 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:34:13.525589 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:34:13.525592 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:34:13.525595 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:34:13.526122 (+ 527us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:34:13.526483 (+ 361us) server_negotiation.cc:1035] Waiting for connection context
0806 09:34:13.526989 (+ 506us) server_negotiation.cc:299] Negotiation successful
0806 09:34:13.527279 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":608,"thread_start_us":350,"threads_started":1}
I20260806 09:34:13.532135 20932 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 09:34:13.535621 19879 raft_consensus.cc:348] T 00000000000000000000000000000000 P 46848e56a85f4383a7c1bf06569fd19b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46848e56a85f4383a7c1bf06569fd19b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37579 } }
I20260806 09:34:13.535995 19879 raft_consensus.cc:374] T 00000000000000000000000000000000 P 46848e56a85f4383a7c1bf06569fd19b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:34:13.536103 19879 raft_consensus.cc:729] T 00000000000000000000000000000000 P 46848e56a85f4383a7c1bf06569fd19b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46848e56a85f4383a7c1bf06569fd19b, State: Initialized, Role: FOLLOWER
I20260806 09:34:13.536854 19879 consensus_queue.cc:260] T 00000000000000000000000000000000 P 46848e56a85f4383a7c1bf06569fd19b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46848e56a85f4383a7c1bf06569fd19b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37579 } }
I20260806 09:34:13.537094 19879 raft_consensus.cc:388] T 00000000000000000000000000000000 P 46848e56a85f4383a7c1bf06569fd19b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 09:34:13.537241 19879 raft_consensus.cc:482] T 00000000000000000000000000000000 P 46848e56a85f4383a7c1bf06569fd19b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 09:34:13.537431 19879 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 46848e56a85f4383a7c1bf06569fd19b [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:34:13.539510 19879 raft_consensus.cc:504] T 00000000000000000000000000000000 P 46848e56a85f4383a7c1bf06569fd19b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46848e56a85f4383a7c1bf06569fd19b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37579 } }
I20260806 09:34:13.540087 19879 leader_election.cc:302] T 00000000000000000000000000000000 P 46848e56a85f4383a7c1bf06569fd19b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 46848e56a85f4383a7c1bf06569fd19b; no voters:
I20260806 09:34:13.541404 19879 leader_election.cc:288] T 00000000000000000000000000000000 P 46848e56a85f4383a7c1bf06569fd19b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 09:34:13.541456 19894 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 46848e56a85f4383a7c1bf06569fd19b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:34:13.542970 19894 raft_consensus.cc:686] T 00000000000000000000000000000000 P 46848e56a85f4383a7c1bf06569fd19b [term 1 LEADER]: Becoming Leader. State: Replica: 46848e56a85f4383a7c1bf06569fd19b, State: Running, Role: LEADER
I20260806 09:34:13.543510 19894 consensus_queue.cc:237] T 00000000000000000000000000000000 P 46848e56a85f4383a7c1bf06569fd19b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46848e56a85f4383a7c1bf06569fd19b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37579 } }
I20260806 09:34:13.543810 19879 sys_catalog.cc:564] T 00000000000000000000000000000000 P 46848e56a85f4383a7c1bf06569fd19b [sys.catalog]: configured and running, proceeding with master startup.
I20260806 09:34:13.546088 19896 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46848e56a85f4383a7c1bf06569fd19b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 46848e56a85f4383a7c1bf06569fd19b. Latest consensus state: current_term: 1 leader_uuid: "46848e56a85f4383a7c1bf06569fd19b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46848e56a85f4383a7c1bf06569fd19b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37579 } } }
I20260806 09:34:13.546475 19896 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46848e56a85f4383a7c1bf06569fd19b [sys.catalog]: This master's current role is: LEADER
I20260806 09:34:13.546998 19895 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46848e56a85f4383a7c1bf06569fd19b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "46848e56a85f4383a7c1bf06569fd19b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46848e56a85f4383a7c1bf06569fd19b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37579 } } }
I20260806 09:34:13.547314 19895 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46848e56a85f4383a7c1bf06569fd19b [sys.catalog]: This master's current role is: LEADER
I20260806 09:34:13.558023 19901 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 09:34:13.561796 19882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:13.536607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.37:42883 (local address 127.0.0.1:37579)
0806 09:34:13.536826 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0806 09:34:13.536834 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 09:34:13.540255 (+ 3421us) server_negotiation.cc:407] Connection header received
0806 09:34:13.540455 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:34:13.540459 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:34:13.540523 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:34:13.540602 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:34:13.542889 (+ 2287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:13.544091 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:13.546332 (+ 2241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:13.547348 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:13.547470 (+ 122us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:34:13.551799 (+ 4329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:34:13.551808 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:34:13.551815 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:34:13.551866 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:34:13.556364 (+ 4498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:34:13.557600 (+ 1236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:34:13.557609 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:34:13.557614 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:34:13.557857 (+ 243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:34:13.558641 (+ 784us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:34:13.558649 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:34:13.558655 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:34:13.559119 (+ 464us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:34:13.559485 (+ 366us) server_negotiation.cc:1035] Waiting for connection context
0806 09:34:13.561012 (+ 1527us) server_negotiation.cc:299] Negotiation successful
0806 09:34:13.561247 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 09:34:13.567207 19901 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 09:34:13.577512 19901 catalog_manager.cc:1347] Generated new cluster ID: 7c5b2ab6eac64dba9d5763cd4470f3dd
I20260806 09:34:13.577627 19901 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 09:34:13.595230 19901 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 09:34:13.596828 19901 catalog_manager.cc:1504] Loading token signing keys...
I20260806 09:34:13.606081 19901 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 46848e56a85f4383a7c1bf06569fd19b: Generated new TSK 0
I20260806 09:34:13.607039 19901 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 09:34:13.623005 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 19767
I20260806 09:34:13.624212 19868 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:34:13.715581 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 19767
W20260806 09:34:13.742391 19746 connection.cc:535] client connection to 127.0.0.1:37579 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37579): Connection reset by peer (error 104)
2026-08-06T09:34:13Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (2379 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786008495763054-20932-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 09:34:13 dist-test-slave-45dx krb5kdc[19930](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 09:34:13 dist-test-slave-45dx krb5kdc[19930](info): set up 2 sockets
Aug 06 09:34:13 dist-test-slave-45dx krb5kdc[19930](info): commencing operation
krb5kdc: starting...
W20260806 09:34:15.028856 20932 mini_kdc.cc:121] Time spent starting KDC: real 1.265s user 0.009s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786008495763054-20932-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786008495763054-20932-0/krb5kdc/test-user.keytab.
Aug 06 09:34:15 dist-test-slave-45dx krb5kdc[19930](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008855, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T09:34:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T09:34:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786008495763054-20932-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786008495763054-20932-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 09:34:15.377542 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786008495763054-20932-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786008495763054-20932-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786008495763054-20932-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42499
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786008495763054-20932-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42499
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:34:15.732729 19950 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:34:15.733019 19950 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:34:15.733194 19950 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:34:15.740206 19950 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 09:34:15.740355 19950 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:34:15.740463 19950 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:34:15.740573 19950 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 09:34:15.740658 19950 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 09:34:15.750233 19950 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786008495763054-20932-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786008495763054-20932-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42499
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786008495763054-20932-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42499
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786008495763054-20932-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:34:15.751539 19950 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:34:15.752912 19950 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:34:15.764245 19957 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:34:15.764494 19959 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:34:15.764523 19963 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:34:15.765514 19950 server_base.cc:1034] running on GCE node
I20260806 09:34:15.766336 19950 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:34:15.767658 19950 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:34:15.768898 19950 hybrid_clock.cc:648] HybridClock initialized: now 1786008855768847 us; error 65 us; skew 500 ppm
I20260806 09:34:15.769246 19950 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:34:15 dist-test-slave-45dx krb5kdc[19930](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008855, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:34:15.772475 19950 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 09:34:15.773907 19950 webserver.cc:460] Webserver started at http://127.0.0.1:43055/ using document root <none> and password file <none>
I20260806 09:34:15.774581 19950 fs_manager.cc:359] Metadata directory not provided
I20260806 09:34:15.774734 19950 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:34:15.775053 19950 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:34:15.778081 19950 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786008495763054-20932-0/minicluster-data/master-0/data/instance:
uuid: "622e6cf9ced649e390f86f8d26603b82"
format_stamp: "Formatted at 2026-08-06 09:34:15 on dist-test-slave-45dx"
server_key: "695869a075d65c8e0656b42cb995b74d"
server_key_iv: "cf47c5dacad228816d56ed23034d4fb8"
server_key_version: "encryptionkey@0"
I20260806 09:34:15.778846 19950 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786008495763054-20932-0/minicluster-data/master-0/wal/instance:
uuid: "622e6cf9ced649e390f86f8d26603b82"
format_stamp: "Formatted at 2026-08-06 09:34:15 on dist-test-slave-45dx"
server_key: "695869a075d65c8e0656b42cb995b74d"
server_key_iv: "cf47c5dacad228816d56ed23034d4fb8"
server_key_version: "encryptionkey@0"
I20260806 09:34:15.784113 19950 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 09:34:15.787569 19975 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:34:15.789005 19950 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 09:34:15.789193 19950 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786008495763054-20932-0/minicluster-data/master-0/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786008495763054-20932-0/minicluster-data/master-0/wal
uuid: "622e6cf9ced649e390f86f8d26603b82"
format_stamp: "Formatted at 2026-08-06 09:34:15 on dist-test-slave-45dx"
server_key: "695869a075d65c8e0656b42cb995b74d"
server_key_iv: "cf47c5dacad228816d56ed23034d4fb8"
server_key_version: "encryptionkey@0"
I20260806 09:34:15.789405 19950 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786008495763054-20932-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786008495763054-20932-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786008495763054-20932-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:34:15.800748 19950 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:34:15.805114 19950 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:34:15.805404 19950 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:34:15.824098 19950 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42499
I20260806 09:34:15.824128 20056 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42499 every 8 connection(s)
I20260806 09:34:15.825477 19950 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
I20260806 09:34:15.829151 20057 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:34:15.832012 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 19950
I20260806 09:34:15.832217 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786008495763054-20932-0/minicluster-data/master-0/wal/instance
I20260806 09:34:15.832830 20932 external_mini_cluster.cc:1428] Setting key 4372438a5ffc76a42c7c9e0693bf9d67
I20260806 09:34:15.841539 20057 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 622e6cf9ced649e390f86f8d26603b82: Bootstrap starting.
I20260806 09:34:15.845362 20057 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 622e6cf9ced649e390f86f8d26603b82: Neither blocks nor log segments found. Creating new log.
I20260806 09:34:15.846586 20057 log.cc:824] T 00000000000000000000000000000000 P 622e6cf9ced649e390f86f8d26603b82: Log is configured to *not* fsync() on all Append() calls
I20260806 09:34:15.846719 20063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:15.834789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.37:36263 (local address 127.0.0.1:42499)
0806 09:34:15.835630 (+ 841us) server_negotiation.cc:206] Beginning negotiation
0806 09:34:15.835642 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 09:34:15.835683 (+ 41us) server_negotiation.cc:407] Connection header received
0806 09:34:15.837397 (+ 1714us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:34:15.837420 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:34:15.837818 (+ 398us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:34:15.838144 (+ 326us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:34:15.838839 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:15.840140 (+ 1301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:15.842030 (+ 1890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:15.842754 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:15.842901 (+ 147us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:34:15.844376 (+ 1475us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:34:15.844386 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:34:15.844400 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:34:15.844951 (+ 551us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:34:15.845045 (+ 94us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:34:15.845071 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:34:15.845203 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0806 09:34:15.845575 (+ 372us) server_negotiation.cc:299] Negotiation successful
0806 09:34:15.845725 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":634,"thread_start_us":425,"threads_started":1}
I20260806 09:34:15.849573 20057 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 622e6cf9ced649e390f86f8d26603b82: No bootstrap required, opened a new log
I20260806 09:34:15.850605 20932 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 09:34:15.857990 20057 raft_consensus.cc:348] T 00000000000000000000000000000000 P 622e6cf9ced649e390f86f8d26603b82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "622e6cf9ced649e390f86f8d26603b82" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42499 } }
I20260806 09:34:15.858358 20057 raft_consensus.cc:374] T 00000000000000000000000000000000 P 622e6cf9ced649e390f86f8d26603b82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:34:15.858510 20057 raft_consensus.cc:729] T 00000000000000000000000000000000 P 622e6cf9ced649e390f86f8d26603b82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 622e6cf9ced649e390f86f8d26603b82, State: Initialized, Role: FOLLOWER
I20260806 09:34:15.859184 20057 consensus_queue.cc:260] T 00000000000000000000000000000000 P 622e6cf9ced649e390f86f8d26603b82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "622e6cf9ced649e390f86f8d26603b82" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42499 } }
I20260806 09:34:15.859483 20057 raft_consensus.cc:388] T 00000000000000000000000000000000 P 622e6cf9ced649e390f86f8d26603b82 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 09:34:15.859669 20057 raft_consensus.cc:482] T 00000000000000000000000000000000 P 622e6cf9ced649e390f86f8d26603b82 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 09:34:15.859858 20057 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 622e6cf9ced649e390f86f8d26603b82 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:34:15.861452 20063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:15.855075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.37:42257 (local address 127.0.0.1:42499)
0806 09:34:15.855266 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0806 09:34:15.855272 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:34:15.855639 (+ 367us) server_negotiation.cc:407] Connection header received
0806 09:34:15.855865 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:34:15.855870 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:34:15.855934 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:34:15.856068 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:34:15.856638 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:15.857398 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:15.858831 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:15.859281 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:15.859430 (+ 149us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:34:15.860673 (+ 1243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:34:15.860678 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:34:15.860681 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:34:15.860700 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:34:15.860758 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:34:15.860780 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:34:15.860920 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0806 09:34:15.861116 (+ 196us) server_negotiation.cc:299] Negotiation successful
0806 09:34:15.861170 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 09:34:15.862996 20057 raft_consensus.cc:504] T 00000000000000000000000000000000 P 622e6cf9ced649e390f86f8d26603b82 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "622e6cf9ced649e390f86f8d26603b82" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42499 } }
I20260806 09:34:15.863535 20057 leader_election.cc:302] T 00000000000000000000000000000000 P 622e6cf9ced649e390f86f8d26603b82 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 622e6cf9ced649e390f86f8d26603b82; no voters:
I20260806 09:34:15.864135 20057 leader_election.cc:288] T 00000000000000000000000000000000 P 622e6cf9ced649e390f86f8d26603b82 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 09:34:15.864267 20073 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 622e6cf9ced649e390f86f8d26603b82 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:34:15.866330 20057 sys_catalog.cc:564] T 00000000000000000000000000000000 P 622e6cf9ced649e390f86f8d26603b82 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 09:34:15.871385 20073 raft_consensus.cc:686] T 00000000000000000000000000000000 P 622e6cf9ced649e390f86f8d26603b82 [term 1 LEADER]: Becoming Leader. State: Replica: 622e6cf9ced649e390f86f8d26603b82, State: Running, Role: LEADER
I20260806 09:34:15.871869 20073 consensus_queue.cc:237] T 00000000000000000000000000000000 P 622e6cf9ced649e390f86f8d26603b82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "622e6cf9ced649e390f86f8d26603b82" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42499 } }
I20260806 09:34:15.874922 20074 sys_catalog.cc:455] T 00000000000000000000000000000000 P 622e6cf9ced649e390f86f8d26603b82 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "622e6cf9ced649e390f86f8d26603b82" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "622e6cf9ced649e390f86f8d26603b82" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42499 } } }
I20260806 09:34:15.875093 20076 sys_catalog.cc:455] T 00000000000000000000000000000000 P 622e6cf9ced649e390f86f8d26603b82 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 622e6cf9ced649e390f86f8d26603b82. Latest consensus state: current_term: 1 leader_uuid: "622e6cf9ced649e390f86f8d26603b82" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "622e6cf9ced649e390f86f8d26603b82" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42499 } } }
I20260806 09:34:15.875396 20074 sys_catalog.cc:458] T 00000000000000000000000000000000 P 622e6cf9ced649e390f86f8d26603b82 [sys.catalog]: This master's current role is: LEADER
I20260806 09:34:15.875393 20076 sys_catalog.cc:458] T 00000000000000000000000000000000 P 622e6cf9ced649e390f86f8d26603b82 [sys.catalog]: This master's current role is: LEADER
W20260806 09:34:15.882766 20095 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 622e6cf9ced649e390f86f8d26603b82: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 09:34:15.882895 20095 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 09:34:15.883045 20097 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 09:34:15.884714 20097 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 09:34:15.892267 20097 catalog_manager.cc:1347] Generated new cluster ID: c1e95d6247f04b17adb0d0e64103fe36
I20260806 09:34:15.892417 20097 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 09:34:15.907514 20097 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 09:34:15.909241 20097 catalog_manager.cc:1504] Loading token signing keys...
I20260806 09:34:15.915584 20097 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 622e6cf9ced649e390f86f8d26603b82: Generated new TSK 0
I20260806 09:34:15.916553 20097 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 09:34:15.925262 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 19950
W20260806 09:34:15.926879 20051 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.37:36263
W20260806 09:34:15.927649 20932 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 19950: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 09:34:15.927807 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 19950
W20260806 09:34:15.954823 19923 connection.cc:535] client connection to 127.0.0.1:42499 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42499): Connection reset by peer (error 104)
2026-08-06T09:34:15Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (2210 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786008495763054-20932-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 09:34:16 dist-test-slave-45dx krb5kdc[20109](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 09:34:16 dist-test-slave-45dx krb5kdc[20109](info): set up 2 sockets
Aug 06 09:34:16 dist-test-slave-45dx krb5kdc[20109](info): commencing operation
krb5kdc: starting...
W20260806 09:34:17.247952 20932 mini_kdc.cc:121] Time spent starting KDC: real 1.273s user 0.007s sys 0.047s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786008495763054-20932-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786008495763054-20932-0/krb5kdc/test-user.keytab.
Aug 06 09:34:17 dist-test-slave-45dx krb5kdc[20109](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008857, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T09:34:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T09:34:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786008495763054-20932-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786008495763054-20932-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 09:34:17.591490 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786008495763054-20932-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786008495763054-20932-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786008495763054-20932-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38649
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786008495763054-20932-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38649
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:34:17.934993 20125 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:34:17.935290 20125 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:34:17.935451 20125 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:34:17.942564 20125 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 09:34:17.942667 20125 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:34:17.942799 20125 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:34:17.942907 20125 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 09:34:17.942970 20125 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 09:34:17.952674 20125 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786008495763054-20932-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786008495763054-20932-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38649
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786008495763054-20932-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38649
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786008495763054-20932-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:34:17.953881 20125 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:34:17.955179 20125 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:34:17.966940 20132 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:34:17.967037 20133 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:34:17.966988 20135 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:34:17.968158 20125 server_base.cc:1034] running on GCE node
I20260806 09:34:17.969000 20125 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:34:17.970294 20125 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:34:17.971529 20125 hybrid_clock.cc:648] HybridClock initialized: now 1786008857971484 us; error 40 us; skew 500 ppm
I20260806 09:34:17.971884 20125 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:34:17 dist-test-slave-45dx krb5kdc[20109](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008857, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:34:17.979074 20125 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 09:34:17.980647 20125 webserver.cc:460] Webserver started at http://127.0.0.1:36689/ using document root <none> and password file <none>
I20260806 09:34:17.981351 20125 fs_manager.cc:359] Metadata directory not provided
I20260806 09:34:17.981503 20125 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:34:17.981842 20125 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:34:17.984898 20125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786008495763054-20932-0/minicluster-data/master-0/data/instance:
uuid: "8d1d44f431654f9b88bc492db7cd793c"
format_stamp: "Formatted at 2026-08-06 09:34:17 on dist-test-slave-45dx"
server_key: "2a69b465b9a169386c77b94de12352c0"
server_key_iv: "5f994985727b89a427cfb557d965d7b4"
server_key_version: "encryptionkey@0"
I20260806 09:34:17.985690 20125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786008495763054-20932-0/minicluster-data/master-0/wal/instance:
uuid: "8d1d44f431654f9b88bc492db7cd793c"
format_stamp: "Formatted at 2026-08-06 09:34:17 on dist-test-slave-45dx"
server_key: "2a69b465b9a169386c77b94de12352c0"
server_key_iv: "5f994985727b89a427cfb557d965d7b4"
server_key_version: "encryptionkey@0"
I20260806 09:34:17.990794 20125 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 09:34:17.994189 20149 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:34:17.995481 20125 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 09:34:17.995657 20125 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786008495763054-20932-0/minicluster-data/master-0/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786008495763054-20932-0/minicluster-data/master-0/wal
uuid: "8d1d44f431654f9b88bc492db7cd793c"
format_stamp: "Formatted at 2026-08-06 09:34:17 on dist-test-slave-45dx"
server_key: "2a69b465b9a169386c77b94de12352c0"
server_key_iv: "5f994985727b89a427cfb557d965d7b4"
server_key_version: "encryptionkey@0"
I20260806 09:34:17.995867 20125 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786008495763054-20932-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786008495763054-20932-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786008495763054-20932-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:34:18.001716 20125 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:34:18.005759 20125 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:34:18.006071 20125 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:34:18.024768 20125 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38649
I20260806 09:34:18.024812 20227 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38649 every 8 connection(s)
I20260806 09:34:18.026209 20125 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
I20260806 09:34:18.029932 20228 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:34:18.033483 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 20125
I20260806 09:34:18.033691 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786008495763054-20932-0/minicluster-data/master-0/wal/instance
I20260806 09:34:18.034324 20932 external_mini_cluster.cc:1428] Setting key 00439e4f938b4312465d9367cb0978ea
I20260806 09:34:18.041976 20232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:18.036191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.37:39551 (local address 127.0.0.1:38649)
0806 09:34:18.036922 (+ 731us) server_negotiation.cc:206] Beginning negotiation
0806 09:34:18.036933 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 09:34:18.036974 (+ 41us) server_negotiation.cc:407] Connection header received
0806 09:34:18.038564 (+ 1590us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:34:18.038602 (+ 38us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:34:18.038978 (+ 376us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:34:18.039379 (+ 401us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:34:18.040442 (+ 1063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:34:18.040448 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:34:18.040460 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:34:18.040476 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:34:18.040530 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:34:18.040553 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:34:18.040777 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0806 09:34:18.040915 (+ 138us) server_negotiation.cc:299] Negotiation successful
0806 09:34:18.041096 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":537,"thread_start_us":301,"threads_started":1}
I20260806 09:34:18.042330 20228 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d1d44f431654f9b88bc492db7cd793c: Bootstrap starting.
I20260806 09:34:18.045548 20932 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 09:34:18.045687 20228 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8d1d44f431654f9b88bc492db7cd793c: Neither blocks nor log segments found. Creating new log.
I20260806 09:34:18.046677 20228 log.cc:824] T 00000000000000000000000000000000 P 8d1d44f431654f9b88bc492db7cd793c: Log is configured to *not* fsync() on all Append() calls
I20260806 09:34:18.049511 20228 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d1d44f431654f9b88bc492db7cd793c: No bootstrap required, opened a new log
I20260806 09:34:18.053066 20232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:18.050395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.37:52161 (local address 127.0.0.1:38649)
0806 09:34:18.050613 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0806 09:34:18.050619 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:34:18.050978 (+ 359us) server_negotiation.cc:407] Connection header received
0806 09:34:18.051146 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:34:18.051150 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:34:18.051193 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:34:18.051279 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:34:18.052280 (+ 1001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:34:18.052284 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:34:18.052287 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:34:18.052365 (+ 78us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:34:18.052403 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:34:18.052419 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:34:18.052512 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 09:34:18.052663 (+ 151us) server_negotiation.cc:299] Negotiation successful
0806 09:34:18.052712 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260806 09:34:18.058246 20228 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8d1d44f431654f9b88bc492db7cd793c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d1d44f431654f9b88bc492db7cd793c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38649 } }
I20260806 09:34:18.058725 20228 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8d1d44f431654f9b88bc492db7cd793c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:34:18.058840 20228 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8d1d44f431654f9b88bc492db7cd793c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d1d44f431654f9b88bc492db7cd793c, State: Initialized, Role: FOLLOWER
I20260806 09:34:18.059432 20228 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8d1d44f431654f9b88bc492db7cd793c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d1d44f431654f9b88bc492db7cd793c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38649 } }
I20260806 09:34:18.059680 20228 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8d1d44f431654f9b88bc492db7cd793c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 09:34:18.059842 20228 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8d1d44f431654f9b88bc492db7cd793c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 09:34:18.060032 20228 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8d1d44f431654f9b88bc492db7cd793c [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:34:18.062363 20228 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8d1d44f431654f9b88bc492db7cd793c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d1d44f431654f9b88bc492db7cd793c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38649 } }
I20260806 09:34:18.062853 20228 leader_election.cc:302] T 00000000000000000000000000000000 P 8d1d44f431654f9b88bc492db7cd793c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8d1d44f431654f9b88bc492db7cd793c; no voters:
I20260806 09:34:18.063438 20228 leader_election.cc:288] T 00000000000000000000000000000000 P 8d1d44f431654f9b88bc492db7cd793c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 09:34:18.063604 20241 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8d1d44f431654f9b88bc492db7cd793c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:34:18.064286 20241 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8d1d44f431654f9b88bc492db7cd793c [term 1 LEADER]: Becoming Leader. State: Replica: 8d1d44f431654f9b88bc492db7cd793c, State: Running, Role: LEADER
I20260806 09:34:18.064880 20241 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8d1d44f431654f9b88bc492db7cd793c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d1d44f431654f9b88bc492db7cd793c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38649 } }
I20260806 09:34:18.065569 20228 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8d1d44f431654f9b88bc492db7cd793c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 09:34:18.067835 20243 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d1d44f431654f9b88bc492db7cd793c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8d1d44f431654f9b88bc492db7cd793c. Latest consensus state: current_term: 1 leader_uuid: "8d1d44f431654f9b88bc492db7cd793c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d1d44f431654f9b88bc492db7cd793c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38649 } } }
I20260806 09:34:18.068347 20243 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d1d44f431654f9b88bc492db7cd793c [sys.catalog]: This master's current role is: LEADER
I20260806 09:34:18.068157 20242 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d1d44f431654f9b88bc492db7cd793c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8d1d44f431654f9b88bc492db7cd793c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d1d44f431654f9b88bc492db7cd793c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38649 } } }
I20260806 09:34:18.068552 20242 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d1d44f431654f9b88bc492db7cd793c [sys.catalog]: This master's current role is: LEADER
I20260806 09:34:18.069437 20253 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 09:34:18.075052 20253 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 09:34:18.083455 20253 catalog_manager.cc:1347] Generated new cluster ID: d69e4b75d4244bd9b4a1cf8b38f74fb1
I20260806 09:34:18.083560 20253 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 09:34:18.098459 20253 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 09:34:18.098822 20253 catalog_manager.cc:1504] Loading token signing keys...
I20260806 09:34:18.108011 20253 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8d1d44f431654f9b88bc492db7cd793c: Generated new TSK 0
I20260806 09:34:18.108773 20253 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 09:34:18.115899 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 20125
W20260806 09:34:18.117116 20223 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.37:39551
W20260806 09:34:18.117718 20932 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 20125: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 09:34:18.117870 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 20125
2026-08-06T09:34:18Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (2188 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786008495763054-20932-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 09:34:18 dist-test-slave-45dx krb5kdc[20269](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 09:34:18 dist-test-slave-45dx krb5kdc[20269](info): set up 2 sockets
Aug 06 09:34:18 dist-test-slave-45dx krb5kdc[20269](info): commencing operation
krb5kdc: starting...
W20260806 09:34:19.432785 20932 mini_kdc.cc:121] Time spent starting KDC: real 1.269s user 0.006s sys 0.049s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786008495763054-20932-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786008495763054-20932-0/krb5kdc/test-user.keytab.
Aug 06 09:34:19 dist-test-slave-45dx krb5kdc[20269](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008859, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T09:34:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T09:34:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.113.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.113.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.113.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab.
Entry for principal kudu/127.20.113.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab.
I20260806 09:34:19.793256 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786008495763054-20932-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786008495763054-20932-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786008495763054-20932-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.113.62:46805
--webserver_interface=127.20.113.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab
--principal=kudu/127.20.113.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:44847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.113.62:46805
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:34:20.144109 20291 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:34:20.144448 20291 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:34:20.144551 20291 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:34:20.151742 20291 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 09:34:20.151854 20291 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:34:20.151901 20291 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:34:20.151962 20291 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 09:34:20.152009 20291 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 09:34:20.161568 20291 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:44847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786008495763054-20932-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786008495763054-20932-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.113.62:46805
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab
--principal=kudu/127.20.113.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.113.62:46805
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.113.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786008495763054-20932-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:34:20.162889 20291 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:34:20.164255 20291 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:34:20.176080 20298 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:34:20.176107 20304 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:34:20.176353 20299 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:34:20.177273 20291 server_base.cc:1034] running on GCE node
I20260806 09:34:20.177975 20291 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:34:20.179243 20291 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:34:20.180414 20291 hybrid_clock.cc:648] HybridClock initialized: now 1786008860180363 us; error 29 us; skew 500 ppm
I20260806 09:34:20.180713 20291 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:34:20 dist-test-slave-45dx krb5kdc[20269](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008860, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:34:20.183764 20291 init.cc:377] Logged in from keytab as kudu/127.20.113.62@KRBTEST.COM (short username kudu)
I20260806 09:34:20.185225 20291 webserver.cc:460] Webserver started at http://127.20.113.62:42109/ using document root <none> and password file <none>
I20260806 09:34:20.185853 20291 fs_manager.cc:359] Metadata directory not provided
I20260806 09:34:20.185984 20291 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:34:20.186298 20291 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:34:20.189251 20291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786008495763054-20932-0/minicluster-data/master-0/data/instance:
uuid: "1e0a8a32e2d04c5abf7b9b7faa027dc6"
format_stamp: "Formatted at 2026-08-06 09:34:20 on dist-test-slave-45dx"
server_key: "171dc43d2570312140764d4b12438dcf"
server_key_iv: "9c85fd5ee82df563e92cd49f97b5cb87"
server_key_version: "encryptionkey@0"
I20260806 09:34:20.190021 20291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786008495763054-20932-0/minicluster-data/master-0/wal/instance:
uuid: "1e0a8a32e2d04c5abf7b9b7faa027dc6"
format_stamp: "Formatted at 2026-08-06 09:34:20 on dist-test-slave-45dx"
server_key: "171dc43d2570312140764d4b12438dcf"
server_key_iv: "9c85fd5ee82df563e92cd49f97b5cb87"
server_key_version: "encryptionkey@0"
I20260806 09:34:20.195281 20291 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 09:34:20.198851 20319 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:34:20.200145 20291 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 09:34:20.200387 20291 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786008495763054-20932-0/minicluster-data/master-0/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786008495763054-20932-0/minicluster-data/master-0/wal
uuid: "1e0a8a32e2d04c5abf7b9b7faa027dc6"
format_stamp: "Formatted at 2026-08-06 09:34:20 on dist-test-slave-45dx"
server_key: "171dc43d2570312140764d4b12438dcf"
server_key_iv: "9c85fd5ee82df563e92cd49f97b5cb87"
server_key_version: "encryptionkey@0"
I20260806 09:34:20.200626 20291 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786008495763054-20932-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786008495763054-20932-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786008495763054-20932-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:34:20.215819 20291 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:34:20.220115 20291 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:34:20.220407 20291 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:34:20.238273 20291 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.62:46805
I20260806 09:34:20.238305 20394 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.62:46805 every 8 connection(s)
I20260806 09:34:20.239549 20291 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
I20260806 09:34:20.243175 20397 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:34:20.247372 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 20291
I20260806 09:34:20.247567 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786008495763054-20932-0/minicluster-data/master-0/wal/instance
I20260806 09:34:20.248155 20932 external_mini_cluster.cc:1428] Setting key 3d37ee170f5a1b0b6a5c67613869a7e5
I20260806 09:34:20.255748 20397 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e0a8a32e2d04c5abf7b9b7faa027dc6: Bootstrap starting.
I20260806 09:34:20.258837 20397 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1e0a8a32e2d04c5abf7b9b7faa027dc6: Neither blocks nor log segments found. Creating new log.
I20260806 09:34:20.259914 20397 log.cc:824] T 00000000000000000000000000000000 P 1e0a8a32e2d04c5abf7b9b7faa027dc6: Log is configured to *not* fsync() on all Append() calls
Aug 06 09:34:20 dist-test-slave-45dx krb5kdc[20269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008859, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.113.62@KRBTEST.COM
I20260806 09:34:20.263599 20397 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e0a8a32e2d04c5abf7b9b7faa027dc6: No bootstrap required, opened a new log
I20260806 09:34:20.275383 20400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:20.249976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57143 (local address 127.20.113.62:46805)
0806 09:34:20.250788 (+ 812us) server_negotiation.cc:206] Beginning negotiation
0806 09:34:20.250800 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 09:34:20.250845 (+ 45us) server_negotiation.cc:407] Connection header received
0806 09:34:20.252546 (+ 1701us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:34:20.252569 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:34:20.252973 (+ 404us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:34:20.253344 (+ 371us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:34:20.254809 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:20.256187 (+ 1378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:20.257483 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:20.258114 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:20.258301 (+ 187us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:34:20.263762 (+ 5461us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:34:20.263775 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:34:20.263794 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:34:20.263840 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:34:20.268591 (+ 4751us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:34:20.269832 (+ 1241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:34:20.269840 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:34:20.269848 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:34:20.270083 (+ 235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:34:20.270814 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:34:20.270819 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:34:20.270823 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:34:20.271636 (+ 813us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:34:20.273148 (+ 1512us) server_negotiation.cc:1035] Waiting for connection context
0806 09:34:20.273494 (+ 346us) server_negotiation.cc:299] Negotiation successful
0806 09:34:20.273880 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":617,"thread_start_us":401,"threads_started":1}
I20260806 09:34:20.276603 20397 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1e0a8a32e2d04c5abf7b9b7faa027dc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e0a8a32e2d04c5abf7b9b7faa027dc6" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 46805 } }
I20260806 09:34:20.277092 20397 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1e0a8a32e2d04c5abf7b9b7faa027dc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:34:20.277245 20397 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1e0a8a32e2d04c5abf7b9b7faa027dc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e0a8a32e2d04c5abf7b9b7faa027dc6, State: Initialized, Role: FOLLOWER
I20260806 09:34:20.278033 20397 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1e0a8a32e2d04c5abf7b9b7faa027dc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e0a8a32e2d04c5abf7b9b7faa027dc6" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 46805 } }
I20260806 09:34:20.278390 20397 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1e0a8a32e2d04c5abf7b9b7faa027dc6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 09:34:20.278582 20397 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1e0a8a32e2d04c5abf7b9b7faa027dc6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 09:34:20.278827 20397 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1e0a8a32e2d04c5abf7b9b7faa027dc6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:34:20.280464 20932 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 09:34:20.282498 20397 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1e0a8a32e2d04c5abf7b9b7faa027dc6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e0a8a32e2d04c5abf7b9b7faa027dc6" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 46805 } }
I20260806 09:34:20.283021 20397 leader_election.cc:302] T 00000000000000000000000000000000 P 1e0a8a32e2d04c5abf7b9b7faa027dc6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1e0a8a32e2d04c5abf7b9b7faa027dc6; no voters:
I20260806 09:34:20.283797 20397 leader_election.cc:288] T 00000000000000000000000000000000 P 1e0a8a32e2d04c5abf7b9b7faa027dc6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 09:34:20.283908 20405 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1e0a8a32e2d04c5abf7b9b7faa027dc6 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:34:20.286582 20397 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1e0a8a32e2d04c5abf7b9b7faa027dc6 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 09:34:20.287818 20405 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1e0a8a32e2d04c5abf7b9b7faa027dc6 [term 1 LEADER]: Becoming Leader. State: Replica: 1e0a8a32e2d04c5abf7b9b7faa027dc6, State: Running, Role: LEADER
I20260806 09:34:20.288614 20405 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1e0a8a32e2d04c5abf7b9b7faa027dc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e0a8a32e2d04c5abf7b9b7faa027dc6" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 46805 } }
I20260806 09:34:20.296695 20412 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e0a8a32e2d04c5abf7b9b7faa027dc6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1e0a8a32e2d04c5abf7b9b7faa027dc6. Latest consensus state: current_term: 1 leader_uuid: "1e0a8a32e2d04c5abf7b9b7faa027dc6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e0a8a32e2d04c5abf7b9b7faa027dc6" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 46805 } } }
I20260806 09:34:20.297060 20412 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e0a8a32e2d04c5abf7b9b7faa027dc6 [sys.catalog]: This master's current role is: LEADER
I20260806 09:34:20.297793 20414 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e0a8a32e2d04c5abf7b9b7faa027dc6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1e0a8a32e2d04c5abf7b9b7faa027dc6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e0a8a32e2d04c5abf7b9b7faa027dc6" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 46805 } } }
I20260806 09:34:20.298046 20414 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e0a8a32e2d04c5abf7b9b7faa027dc6 [sys.catalog]: This master's current role is: LEADER
W20260806 09:34:20.304234 20439 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1e0a8a32e2d04c5abf7b9b7faa027dc6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 09:34:20.304425 20439 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 09:34:20.304586 20435 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 09:34:20.307037 20435 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 09:34:20.309129 20400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:20.290092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34375 (local address 127.20.113.62:46805)
0806 09:34:20.290288 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0806 09:34:20.290296 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 09:34:20.291526 (+ 1230us) server_negotiation.cc:407] Connection header received
0806 09:34:20.291921 (+ 395us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:34:20.291930 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:34:20.292049 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:34:20.292148 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:34:20.295181 (+ 3033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:20.296244 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:20.298261 (+ 2017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:20.299054 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:20.299210 (+ 156us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:34:20.302154 (+ 2944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:34:20.302159 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:34:20.302162 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:34:20.302191 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:34:20.305704 (+ 3513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:34:20.306688 (+ 984us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:34:20.306693 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:34:20.306697 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:34:20.306834 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:34:20.307387 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:34:20.307396 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:34:20.307399 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:34:20.307729 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:34:20.308009 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0806 09:34:20.308446 (+ 437us) server_negotiation.cc:299] Negotiation successful
0806 09:34:20.308622 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 09:34:20.316525 20435 catalog_manager.cc:1347] Generated new cluster ID: eca6fba7755e4cc198d331851e5c2adc
I20260806 09:34:20.316636 20435 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 09:34:20.329931 20435 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 09:34:20.331373 20435 catalog_manager.cc:1504] Loading token signing keys...
I20260806 09:34:20.336431 20435 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1e0a8a32e2d04c5abf7b9b7faa027dc6: Generated new TSK 0
I20260806 09:34:20.337586 20435 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 09:34:20.345661 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 20291
I20260806 09:34:20.347177 20387 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:34:20.435110 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 20291
W20260806 09:34:20.461275 20267 connection.cc:535] client connection to 127.20.113.62:46805 recv error: Network error: failed to read from TLS socket (remote: 127.20.113.62:46805): Connection reset by peer (error 104)
2026-08-06T09:34:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (2316 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786008495763054-20932-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 09:34:20 dist-test-slave-45dx krb5kdc[20449](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 09:34:20 dist-test-slave-45dx krb5kdc[20449](info): set up 2 sockets
Aug 06 09:34:20 dist-test-slave-45dx krb5kdc[20449](info): commencing operation
krb5kdc: starting...
W20260806 09:34:21.755959 20932 mini_kdc.cc:121] Time spent starting KDC: real 1.276s user 0.000s sys 0.058s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786008495763054-20932-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786008495763054-20932-0/krb5kdc/test-user.keytab.
Aug 06 09:34:21 dist-test-slave-45dx krb5kdc[20449](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008861, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T09:34:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T09:34:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.113.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.113.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.113.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab.
Entry for principal kudu/127.20.113.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab.
I20260806 09:34:22.128858 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786008495763054-20932-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786008495763054-20932-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786008495763054-20932-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.113.62:41981
--webserver_interface=127.20.113.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab
--principal=kudu/127.20.113.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:37813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.113.62:41981
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:34:22.487116 20468 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:34:22.487425 20468 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:34:22.487589 20468 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:34:22.494827 20468 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 09:34:22.494962 20468 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:34:22.495067 20468 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:34:22.495172 20468 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 09:34:22.495254 20468 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 09:34:22.505012 20468 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:37813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786008495763054-20932-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786008495763054-20932-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.113.62:41981
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab
--principal=kudu/127.20.113.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.113.62:41981
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.113.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786008495763054-20932-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:34:22.506353 20468 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:34:22.507709 20468 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:34:22.519719 20476 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:34:22.519799 20477 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:34:22.519768 20484 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:34:22.521006 20468 server_base.cc:1034] running on GCE node
I20260806 09:34:22.521915 20468 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:34:22.523352 20468 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:34:22.524590 20468 hybrid_clock.cc:648] HybridClock initialized: now 1786008862524556 us; error 60 us; skew 500 ppm
I20260806 09:34:22.524955 20468 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:34:22 dist-test-slave-45dx krb5kdc[20449](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008862, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:34:22.528241 20468 init.cc:377] Logged in from keytab as kudu/127.20.113.62@KRBTEST.COM (short username kudu)
I20260806 09:34:22.529832 20468 webserver.cc:460] Webserver started at http://127.20.113.62:34335/ using document root <none> and password file <none>
I20260806 09:34:22.530544 20468 fs_manager.cc:359] Metadata directory not provided
I20260806 09:34:22.530694 20468 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:34:22.531032 20468 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:34:22.534127 20468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786008495763054-20932-0/minicluster-data/master-0/data/instance:
uuid: "b406a361232c460498260127e5c6d1d9"
format_stamp: "Formatted at 2026-08-06 09:34:22 on dist-test-slave-45dx"
server_key: "7edaf86a194f9c99ae275de83db3e3ec"
server_key_iv: "5806edb8f64dc76ba937a9bc1256d520"
server_key_version: "encryptionkey@0"
I20260806 09:34:22.535010 20468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786008495763054-20932-0/minicluster-data/master-0/wal/instance:
uuid: "b406a361232c460498260127e5c6d1d9"
format_stamp: "Formatted at 2026-08-06 09:34:22 on dist-test-slave-45dx"
server_key: "7edaf86a194f9c99ae275de83db3e3ec"
server_key_iv: "5806edb8f64dc76ba937a9bc1256d520"
server_key_version: "encryptionkey@0"
I20260806 09:34:22.540583 20468 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.006s
I20260806 09:34:22.544117 20493 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:34:22.545545 20468 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 09:34:22.545758 20468 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786008495763054-20932-0/minicluster-data/master-0/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786008495763054-20932-0/minicluster-data/master-0/wal
uuid: "b406a361232c460498260127e5c6d1d9"
format_stamp: "Formatted at 2026-08-06 09:34:22 on dist-test-slave-45dx"
server_key: "7edaf86a194f9c99ae275de83db3e3ec"
server_key_iv: "5806edb8f64dc76ba937a9bc1256d520"
server_key_version: "encryptionkey@0"
I20260806 09:34:22.545980 20468 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786008495763054-20932-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786008495763054-20932-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786008495763054-20932-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:34:22.589898 20468 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:34:22.594543 20468 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:34:22.594847 20468 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:34:22.614069 20468 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.62:41981
I20260806 09:34:22.614094 20576 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.62:41981 every 8 connection(s)
I20260806 09:34:22.615466 20468 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
I20260806 09:34:22.619469 20577 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:34:22.623950 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 20468
I20260806 09:34:22.624145 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786008495763054-20932-0/minicluster-data/master-0/wal/instance
I20260806 09:34:22.624765 20932 external_mini_cluster.cc:1428] Setting key 54f0d2403365b6b3840d77c21799c9c6
I20260806 09:34:22.631485 20577 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b406a361232c460498260127e5c6d1d9: Bootstrap starting.
I20260806 09:34:22.635311 20577 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b406a361232c460498260127e5c6d1d9: Neither blocks nor log segments found. Creating new log.
I20260806 09:34:22.636790 20577 log.cc:824] T 00000000000000000000000000000000 P b406a361232c460498260127e5c6d1d9: Log is configured to *not* fsync() on all Append() calls
I20260806 09:34:22.639771 20580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:22.626660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36317 (local address 127.20.113.62:41981)
0806 09:34:22.627503 (+ 843us) server_negotiation.cc:206] Beginning negotiation
0806 09:34:22.627515 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 09:34:22.627558 (+ 43us) server_negotiation.cc:407] Connection header received
0806 09:34:22.629917 (+ 2359us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:34:22.629941 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:34:22.630318 (+ 377us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:34:22.630643 (+ 325us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:34:22.631299 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:22.632755 (+ 1456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:22.634692 (+ 1937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:22.635522 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:22.635669 (+ 147us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:34:22.637135 (+ 1466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:34:22.637143 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:34:22.637157 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:34:22.637963 (+ 806us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:34:22.638075 (+ 112us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:34:22.638111 (+ 36us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:34:22.638289 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0806 09:34:22.638623 (+ 334us) server_negotiation.cc:299] Negotiation successful
0806 09:34:22.638817 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":622,"thread_start_us":404,"threads_started":1}
I20260806 09:34:22.640132 20577 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b406a361232c460498260127e5c6d1d9: No bootstrap required, opened a new log
I20260806 09:34:22.643692 20932 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 09:34:22.650394 20577 raft_consensus.cc:348] T 00000000000000000000000000000000 P b406a361232c460498260127e5c6d1d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b406a361232c460498260127e5c6d1d9" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 41981 } }
I20260806 09:34:22.650933 20577 raft_consensus.cc:374] T 00000000000000000000000000000000 P b406a361232c460498260127e5c6d1d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:34:22.651054 20577 raft_consensus.cc:729] T 00000000000000000000000000000000 P b406a361232c460498260127e5c6d1d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b406a361232c460498260127e5c6d1d9, State: Initialized, Role: FOLLOWER
I20260806 09:34:22.651719 20577 consensus_queue.cc:260] T 00000000000000000000000000000000 P b406a361232c460498260127e5c6d1d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b406a361232c460498260127e5c6d1d9" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 41981 } }
I20260806 09:34:22.652029 20577 raft_consensus.cc:388] T 00000000000000000000000000000000 P b406a361232c460498260127e5c6d1d9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 09:34:22.652196 20577 raft_consensus.cc:482] T 00000000000000000000000000000000 P b406a361232c460498260127e5c6d1d9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 09:34:22.652408 20577 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b406a361232c460498260127e5c6d1d9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:34:22.655225 20577 raft_consensus.cc:504] T 00000000000000000000000000000000 P b406a361232c460498260127e5c6d1d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b406a361232c460498260127e5c6d1d9" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 41981 } }
I20260806 09:34:22.655862 20577 leader_election.cc:302] T 00000000000000000000000000000000 P b406a361232c460498260127e5c6d1d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b406a361232c460498260127e5c6d1d9; no voters:
I20260806 09:34:22.656176 20580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:22.649080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40319 (local address 127.20.113.62:41981)
0806 09:34:22.649290 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0806 09:34:22.649297 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:34:22.649740 (+ 443us) server_negotiation.cc:407] Connection header received
0806 09:34:22.649877 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:34:22.649882 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:34:22.649966 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:34:22.650079 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:34:22.650650 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:22.651825 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:22.653348 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:22.654041 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:22.654181 (+ 140us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:34:22.655317 (+ 1136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:34:22.655326 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:34:22.655331 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:34:22.655352 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:34:22.655401 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:34:22.655423 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:34:22.655536 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0806 09:34:22.655769 (+ 233us) server_negotiation.cc:299] Negotiation successful
0806 09:34:22.655823 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 09:34:22.656665 20577 leader_election.cc:288] T 00000000000000000000000000000000 P b406a361232c460498260127e5c6d1d9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 09:34:22.656778 20592 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b406a361232c460498260127e5c6d1d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:34:22.658679 20577 sys_catalog.cc:564] T 00000000000000000000000000000000 P b406a361232c460498260127e5c6d1d9 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 09:34:22.664206 20592 raft_consensus.cc:686] T 00000000000000000000000000000000 P b406a361232c460498260127e5c6d1d9 [term 1 LEADER]: Becoming Leader. State: Replica: b406a361232c460498260127e5c6d1d9, State: Running, Role: LEADER
I20260806 09:34:22.665247 20592 consensus_queue.cc:237] T 00000000000000000000000000000000 P b406a361232c460498260127e5c6d1d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b406a361232c460498260127e5c6d1d9" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 41981 } }
I20260806 09:34:22.668550 20594 sys_catalog.cc:455] T 00000000000000000000000000000000 P b406a361232c460498260127e5c6d1d9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b406a361232c460498260127e5c6d1d9. Latest consensus state: current_term: 1 leader_uuid: "b406a361232c460498260127e5c6d1d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b406a361232c460498260127e5c6d1d9" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 41981 } } }
I20260806 09:34:22.668982 20594 sys_catalog.cc:458] T 00000000000000000000000000000000 P b406a361232c460498260127e5c6d1d9 [sys.catalog]: This master's current role is: LEADER
I20260806 09:34:22.669499 20593 sys_catalog.cc:455] T 00000000000000000000000000000000 P b406a361232c460498260127e5c6d1d9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b406a361232c460498260127e5c6d1d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b406a361232c460498260127e5c6d1d9" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 41981 } } }
I20260806 09:34:22.669971 20593 sys_catalog.cc:458] T 00000000000000000000000000000000 P b406a361232c460498260127e5c6d1d9 [sys.catalog]: This master's current role is: LEADER
W20260806 09:34:22.675940 20618 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b406a361232c460498260127e5c6d1d9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 09:34:22.676074 20618 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 09:34:22.676182 20617 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 09:34:22.678390 20617 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 09:34:22.687126 20617 catalog_manager.cc:1347] Generated new cluster ID: 9ac3d757699c4b41935084abbc44317c
I20260806 09:34:22.687240 20617 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 09:34:22.697453 20617 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 09:34:22.698678 20617 catalog_manager.cc:1504] Loading token signing keys...
I20260806 09:34:22.710368 20617 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b406a361232c460498260127e5c6d1d9: Generated new TSK 0
I20260806 09:34:22.711428 20617 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 09:34:22.728102 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 20468
W20260806 09:34:22.729313 20568 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:36317
W20260806 09:34:22.730127 20932 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 20468: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 09:34:22.730288 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 20468
W20260806 09:34:22.756734 20447 connection.cc:535] client connection to 127.20.113.62:41981 recv error: Network error: failed to read from TLS socket (remote: 127.20.113.62:41981): Connection reset by peer (error 104)
2026-08-06T09:34:22Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (2297 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786008495763054-20932-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 09:34:22 dist-test-slave-45dx krb5kdc[20627](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 09:34:22 dist-test-slave-45dx krb5kdc[20627](info): set up 2 sockets
Aug 06 09:34:22 dist-test-slave-45dx krb5kdc[20627](info): commencing operation
krb5kdc: starting...
W20260806 09:34:24.071389 20932 mini_kdc.cc:121] Time spent starting KDC: real 1.293s user 0.005s sys 0.058s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786008495763054-20932-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786008495763054-20932-0/krb5kdc/test-user.keytab.
Aug 06 09:34:24 dist-test-slave-45dx krb5kdc[20627](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008864, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T09:34:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T09:34:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.113.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.113.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.113.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab.
Entry for principal kudu/127.20.113.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab.
I20260806 09:34:24.409647 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786008495763054-20932-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786008495763054-20932-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786008495763054-20932-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.113.62:40237
--webserver_interface=127.20.113.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab
--principal=kudu/127.20.113.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:44613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.113.62:40237
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:34:24.748557 20645 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:34:24.748847 20645 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:34:24.749017 20645 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:34:24.755966 20645 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 09:34:24.756060 20645 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:34:24.756171 20645 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:34:24.756271 20645 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 09:34:24.756364 20645 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 09:34:24.765650 20645 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:44613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786008495763054-20932-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786008495763054-20932-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.113.62:40237
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab
--principal=kudu/127.20.113.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.113.62:40237
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.113.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786008495763054-20932-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:34:24.766891 20645 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:34:24.768190 20645 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:34:24.779671 20652 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:34:24.779620 20654 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:34:24.780128 20651 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:34:24.781620 20645 server_base.cc:1034] running on GCE node
I20260806 09:34:24.782388 20645 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:34:24.783610 20645 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:34:24.784868 20645 hybrid_clock.cc:648] HybridClock initialized: now 1786008864784792 us; error 47 us; skew 500 ppm
I20260806 09:34:24.785216 20645 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:34:24 dist-test-slave-45dx krb5kdc[20627](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008864, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:34:24.791472 20645 init.cc:377] Logged in from keytab as kudu/127.20.113.62@KRBTEST.COM (short username kudu)
I20260806 09:34:24.793007 20645 webserver.cc:460] Webserver started at http://127.20.113.62:40965/ using document root <none> and password file <none>
I20260806 09:34:24.793888 20645 fs_manager.cc:359] Metadata directory not provided
I20260806 09:34:24.794045 20645 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:34:24.794369 20645 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:34:24.797318 20645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786008495763054-20932-0/minicluster-data/master-0/data/instance:
uuid: "d3dec4b30a454c33893523069addf7ce"
format_stamp: "Formatted at 2026-08-06 09:34:24 on dist-test-slave-45dx"
server_key: "782f0845515dba06e33aff1b9ad80f37"
server_key_iv: "a78d123cc9b47bb80d69e108be114240"
server_key_version: "encryptionkey@0"
I20260806 09:34:24.798094 20645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786008495763054-20932-0/minicluster-data/master-0/wal/instance:
uuid: "d3dec4b30a454c33893523069addf7ce"
format_stamp: "Formatted at 2026-08-06 09:34:24 on dist-test-slave-45dx"
server_key: "782f0845515dba06e33aff1b9ad80f37"
server_key_iv: "a78d123cc9b47bb80d69e108be114240"
server_key_version: "encryptionkey@0"
I20260806 09:34:24.803499 20645 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 09:34:24.806856 20666 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:34:24.808157 20645 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 09:34:24.808404 20645 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786008495763054-20932-0/minicluster-data/master-0/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786008495763054-20932-0/minicluster-data/master-0/wal
uuid: "d3dec4b30a454c33893523069addf7ce"
format_stamp: "Formatted at 2026-08-06 09:34:24 on dist-test-slave-45dx"
server_key: "782f0845515dba06e33aff1b9ad80f37"
server_key_iv: "a78d123cc9b47bb80d69e108be114240"
server_key_version: "encryptionkey@0"
I20260806 09:34:24.808600 20645 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786008495763054-20932-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786008495763054-20932-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786008495763054-20932-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:34:24.814328 20645 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:34:24.818445 20645 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:34:24.818742 20645 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:34:24.837479 20645 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.62:40237
I20260806 09:34:24.837556 20746 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.62:40237 every 8 connection(s)
I20260806 09:34:24.838786 20645 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
I20260806 09:34:24.840524 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 20645
I20260806 09:34:24.840706 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786008495763054-20932-0/minicluster-data/master-0/wal/instance
I20260806 09:34:24.841286 20932 external_mini_cluster.cc:1428] Setting key 5205226f7b77902cc910d531b0f2251d
I20260806 09:34:24.843221 20747 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:34:24.849165 20752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:24.843056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54973 (local address 127.20.113.62:40237)
0806 09:34:24.843830 (+ 774us) server_negotiation.cc:206] Beginning negotiation
0806 09:34:24.843846 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 09:34:24.843886 (+ 40us) server_negotiation.cc:407] Connection header received
0806 09:34:24.845642 (+ 1756us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:34:24.845662 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:34:24.846007 (+ 345us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:34:24.846420 (+ 413us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:34:24.847644 (+ 1224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:34:24.847650 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:34:24.847665 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:34:24.847688 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:34:24.847757 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:34:24.847776 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:34:24.847951 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0806 09:34:24.848074 (+ 123us) server_negotiation.cc:299] Negotiation successful
0806 09:34:24.848258 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":581,"thread_start_us":372,"threads_started":1}
I20260806 09:34:24.852777 20932 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 09:34:24.856559 20747 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3dec4b30a454c33893523069addf7ce: Bootstrap starting.
I20260806 09:34:24.860090 20747 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d3dec4b30a454c33893523069addf7ce: Neither blocks nor log segments found. Creating new log.
I20260806 09:34:24.860672 20752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:24.858032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53371 (local address 127.20.113.62:40237)
0806 09:34:24.858234 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0806 09:34:24.858241 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:34:24.858736 (+ 495us) server_negotiation.cc:407] Connection header received
0806 09:34:24.858856 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:34:24.858860 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:34:24.858932 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:34:24.859029 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:34:24.859901 (+ 872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:34:24.859909 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:34:24.859916 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:34:24.859937 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:34:24.859990 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:34:24.860009 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:34:24.860140 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0806 09:34:24.860279 (+ 139us) server_negotiation.cc:299] Negotiation successful
0806 09:34:24.860354 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 09:34:24.861444 20747 log.cc:824] T 00000000000000000000000000000000 P d3dec4b30a454c33893523069addf7ce: Log is configured to *not* fsync() on all Append() calls
I20260806 09:34:24.864045 20747 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3dec4b30a454c33893523069addf7ce: No bootstrap required, opened a new log
I20260806 09:34:24.871369 20747 raft_consensus.cc:348] T 00000000000000000000000000000000 P d3dec4b30a454c33893523069addf7ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3dec4b30a454c33893523069addf7ce" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 40237 } }
I20260806 09:34:24.871687 20747 raft_consensus.cc:374] T 00000000000000000000000000000000 P d3dec4b30a454c33893523069addf7ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:34:24.871765 20747 raft_consensus.cc:729] T 00000000000000000000000000000000 P d3dec4b30a454c33893523069addf7ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3dec4b30a454c33893523069addf7ce, State: Initialized, Role: FOLLOWER
I20260806 09:34:24.872221 20747 consensus_queue.cc:260] T 00000000000000000000000000000000 P d3dec4b30a454c33893523069addf7ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3dec4b30a454c33893523069addf7ce" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 40237 } }
I20260806 09:34:24.872529 20747 raft_consensus.cc:388] T 00000000000000000000000000000000 P d3dec4b30a454c33893523069addf7ce [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 09:34:24.872690 20747 raft_consensus.cc:482] T 00000000000000000000000000000000 P d3dec4b30a454c33893523069addf7ce [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 09:34:24.872860 20747 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d3dec4b30a454c33893523069addf7ce [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:34:24.875051 20747 raft_consensus.cc:504] T 00000000000000000000000000000000 P d3dec4b30a454c33893523069addf7ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3dec4b30a454c33893523069addf7ce" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 40237 } }
I20260806 09:34:24.875558 20747 leader_election.cc:302] T 00000000000000000000000000000000 P d3dec4b30a454c33893523069addf7ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d3dec4b30a454c33893523069addf7ce; no voters:
I20260806 09:34:24.876109 20747 leader_election.cc:288] T 00000000000000000000000000000000 P d3dec4b30a454c33893523069addf7ce [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 09:34:24.876255 20766 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d3dec4b30a454c33893523069addf7ce [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:34:24.877053 20766 raft_consensus.cc:686] T 00000000000000000000000000000000 P d3dec4b30a454c33893523069addf7ce [term 1 LEADER]: Becoming Leader. State: Replica: d3dec4b30a454c33893523069addf7ce, State: Running, Role: LEADER
I20260806 09:34:24.877578 20766 consensus_queue.cc:237] T 00000000000000000000000000000000 P d3dec4b30a454c33893523069addf7ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3dec4b30a454c33893523069addf7ce" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 40237 } }
I20260806 09:34:24.878198 20747 sys_catalog.cc:564] T 00000000000000000000000000000000 P d3dec4b30a454c33893523069addf7ce [sys.catalog]: configured and running, proceeding with master startup.
I20260806 09:34:24.880885 20768 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3dec4b30a454c33893523069addf7ce [sys.catalog]: SysCatalogTable state changed. Reason: New leader d3dec4b30a454c33893523069addf7ce. Latest consensus state: current_term: 1 leader_uuid: "d3dec4b30a454c33893523069addf7ce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3dec4b30a454c33893523069addf7ce" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 40237 } } }
I20260806 09:34:24.881305 20768 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3dec4b30a454c33893523069addf7ce [sys.catalog]: This master's current role is: LEADER
I20260806 09:34:24.881711 20767 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3dec4b30a454c33893523069addf7ce [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d3dec4b30a454c33893523069addf7ce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3dec4b30a454c33893523069addf7ce" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 40237 } } }
I20260806 09:34:24.882121 20767 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3dec4b30a454c33893523069addf7ce [sys.catalog]: This master's current role is: LEADER
I20260806 09:34:24.882414 20784 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 09:34:24.888931 20784 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 09:34:24.897480 20784 catalog_manager.cc:1347] Generated new cluster ID: e08b41b7af81442c911a20849796f8f6
I20260806 09:34:24.897598 20784 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 09:34:24.909862 20784 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 09:34:24.910259 20784 catalog_manager.cc:1504] Loading token signing keys...
I20260806 09:34:24.918632 20784 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d3dec4b30a454c33893523069addf7ce: Generated new TSK 0
I20260806 09:34:24.919512 20784 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 09:34:24.936010 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 20645
W20260806 09:34:24.937136 20741 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:54973
W20260806 09:34:24.937821 20932 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 20645: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 09:34:24.937983 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 20645
2026-08-06T09:34:24Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (2204 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786008495763054-20932-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 09:34:25 dist-test-slave-45dx krb5kdc[20801](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 09:34:25 dist-test-slave-45dx krb5kdc[20801](info): set up 2 sockets
Aug 06 09:34:25 dist-test-slave-45dx krb5kdc[20801](info): commencing operation
krb5kdc: starting...
W20260806 09:34:26.257748 20932 mini_kdc.cc:121] Time spent starting KDC: real 1.276s user 0.000s sys 0.057s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786008495763054-20932-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786008495763054-20932-0/krb5kdc/test-user.keytab.
Aug 06 09:34:26 dist-test-slave-45dx krb5kdc[20801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008866, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T09:34:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T09:34:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.113.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.113.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.113.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab.
Entry for principal kudu/127.20.113.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab.
I20260806 09:34:26.647850 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786008495763054-20932-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786008495763054-20932-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786008495763054-20932-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.113.62:35897
--webserver_interface=127.20.113.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab
--principal=kudu/127.20.113.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:35365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.113.62:35897
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:34:26.995873 20824 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:34:26.996186 20824 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:34:26.996277 20824 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:34:27.003530 20824 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 09:34:27.003647 20824 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:34:27.003695 20824 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:34:27.003759 20824 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 09:34:27.003804 20824 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 09:34:27.013904 20824 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:35365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786008495763054-20932-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786008495763054-20932-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.113.62:35897
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab
--principal=kudu/127.20.113.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.113.62:35897
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.113.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786008495763054-20932-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:34:27.015259 20824 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:34:27.016587 20824 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:34:27.028792 20839 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:34:27.028847 20837 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:34:27.028859 20834 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:34:27.029552 20824 server_base.cc:1034] running on GCE node
I20260806 09:34:27.030445 20824 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:34:27.031831 20824 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:34:27.033104 20824 hybrid_clock.cc:648] HybridClock initialized: now 1786008867033051 us; error 57 us; skew 500 ppm
I20260806 09:34:27.033598 20824 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:34:27 dist-test-slave-45dx krb5kdc[20801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008867, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:34:27.036906 20824 init.cc:377] Logged in from keytab as kudu/127.20.113.62@KRBTEST.COM (short username kudu)
I20260806 09:34:27.038322 20824 webserver.cc:460] Webserver started at http://127.20.113.62:38949/ using document root <none> and password file <none>
I20260806 09:34:27.039002 20824 fs_manager.cc:359] Metadata directory not provided
I20260806 09:34:27.039148 20824 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:34:27.039453 20824 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:34:27.042531 20824 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786008495763054-20932-0/minicluster-data/master-0/data/instance:
uuid: "fcc64a3087594f0ebdbd89c0c4f82f56"
format_stamp: "Formatted at 2026-08-06 09:34:27 on dist-test-slave-45dx"
server_key: "8ca20794d1c6e9e93870a3aa555dc890"
server_key_iv: "dc11c8e4c7fac01a4ee54b8a9e97de2d"
server_key_version: "encryptionkey@0"
I20260806 09:34:27.043315 20824 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786008495763054-20932-0/minicluster-data/master-0/wal/instance:
uuid: "fcc64a3087594f0ebdbd89c0c4f82f56"
format_stamp: "Formatted at 2026-08-06 09:34:27 on dist-test-slave-45dx"
server_key: "8ca20794d1c6e9e93870a3aa555dc890"
server_key_iv: "dc11c8e4c7fac01a4ee54b8a9e97de2d"
server_key_version: "encryptionkey@0"
I20260806 09:34:27.048418 20824 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260806 09:34:27.051970 20849 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:34:27.053339 20824 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 09:34:27.053526 20824 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786008495763054-20932-0/minicluster-data/master-0/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786008495763054-20932-0/minicluster-data/master-0/wal
uuid: "fcc64a3087594f0ebdbd89c0c4f82f56"
format_stamp: "Formatted at 2026-08-06 09:34:27 on dist-test-slave-45dx"
server_key: "8ca20794d1c6e9e93870a3aa555dc890"
server_key_iv: "dc11c8e4c7fac01a4ee54b8a9e97de2d"
server_key_version: "encryptionkey@0"
I20260806 09:34:27.053735 20824 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786008495763054-20932-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786008495763054-20932-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786008495763054-20932-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:34:27.067180 20824 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:34:27.071055 20824 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:34:27.071377 20824 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:34:27.089712 20824 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.62:35897
I20260806 09:34:27.089745 20929 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.62:35897 every 8 connection(s)
I20260806 09:34:27.091033 20824 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
I20260806 09:34:27.094705 20930 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:34:27.095599 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 20824
I20260806 09:34:27.095794 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786008495763054-20932-0/minicluster-data/master-0/wal/instance
I20260806 09:34:27.096217 20932 external_mini_cluster.cc:1428] Setting key a6882dbefbecc3c3125a89807f77e2ba
I20260806 09:34:27.108734 20930 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fcc64a3087594f0ebdbd89c0c4f82f56: Bootstrap starting.
Aug 06 09:34:27 dist-test-slave-45dx krb5kdc[20801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008866, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.113.62@KRBTEST.COM
I20260806 09:34:27.111892 20930 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fcc64a3087594f0ebdbd89c0c4f82f56: Neither blocks nor log segments found. Creating new log.
I20260806 09:34:27.113392 20930 log.cc:824] T 00000000000000000000000000000000 P fcc64a3087594f0ebdbd89c0c4f82f56: Log is configured to *not* fsync() on all Append() calls
I20260806 09:34:27.116751 20930 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fcc64a3087594f0ebdbd89c0c4f82f56: No bootstrap required, opened a new log
I20260806 09:34:27.120776 20942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:27.098079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46377 (local address 127.20.113.62:35897)
0806 09:34:27.098896 (+ 817us) server_negotiation.cc:206] Beginning negotiation
0806 09:34:27.098916 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 09:34:27.098964 (+ 48us) server_negotiation.cc:407] Connection header received
0806 09:34:27.101014 (+ 2050us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:34:27.101041 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:34:27.101561 (+ 520us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:34:27.102023 (+ 462us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:34:27.103351 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:27.105413 (+ 2062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:27.106756 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:27.107779 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:27.107950 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:34:27.112277 (+ 4327us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:34:27.112288 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:34:27.112336 (+ 48us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:34:27.112401 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:34:27.116362 (+ 3961us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:34:27.117052 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:34:27.117060 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:34:27.117067 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:34:27.117304 (+ 237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:34:27.117704 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:34:27.117709 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:34:27.117712 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:34:27.118382 (+ 670us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:34:27.118690 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0806 09:34:27.119000 (+ 310us) server_negotiation.cc:299] Negotiation successful
0806 09:34:27.119430 (+ 430us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":568,"thread_start_us":332,"threads_started":1}
I20260806 09:34:27.124607 20932 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 09:34:27.127117 20930 raft_consensus.cc:348] T 00000000000000000000000000000000 P fcc64a3087594f0ebdbd89c0c4f82f56 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcc64a3087594f0ebdbd89c0c4f82f56" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 35897 } }
I20260806 09:34:27.127560 20930 raft_consensus.cc:374] T 00000000000000000000000000000000 P fcc64a3087594f0ebdbd89c0c4f82f56 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:34:27.127671 20930 raft_consensus.cc:729] T 00000000000000000000000000000000 P fcc64a3087594f0ebdbd89c0c4f82f56 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcc64a3087594f0ebdbd89c0c4f82f56, State: Initialized, Role: FOLLOWER
I20260806 09:34:27.128403 20930 consensus_queue.cc:260] T 00000000000000000000000000000000 P fcc64a3087594f0ebdbd89c0c4f82f56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcc64a3087594f0ebdbd89c0c4f82f56" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 35897 } }
I20260806 09:34:27.128727 20930 raft_consensus.cc:388] T 00000000000000000000000000000000 P fcc64a3087594f0ebdbd89c0c4f82f56 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 09:34:27.128927 20930 raft_consensus.cc:482] T 00000000000000000000000000000000 P fcc64a3087594f0ebdbd89c0c4f82f56 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 09:34:27.129110 20930 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fcc64a3087594f0ebdbd89c0c4f82f56 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:34:27.131934 20930 raft_consensus.cc:504] T 00000000000000000000000000000000 P fcc64a3087594f0ebdbd89c0c4f82f56 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcc64a3087594f0ebdbd89c0c4f82f56" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 35897 } }
I20260806 09:34:27.132505 20930 leader_election.cc:302] T 00000000000000000000000000000000 P fcc64a3087594f0ebdbd89c0c4f82f56 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fcc64a3087594f0ebdbd89c0c4f82f56; no voters:
I20260806 09:34:27.133154 20930 leader_election.cc:288] T 00000000000000000000000000000000 P fcc64a3087594f0ebdbd89c0c4f82f56 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 09:34:27.133315 20953 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fcc64a3087594f0ebdbd89c0c4f82f56 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:34:27.134006 20953 raft_consensus.cc:686] T 00000000000000000000000000000000 P fcc64a3087594f0ebdbd89c0c4f82f56 [term 1 LEADER]: Becoming Leader. State: Replica: fcc64a3087594f0ebdbd89c0c4f82f56, State: Running, Role: LEADER
I20260806 09:34:27.134657 20953 consensus_queue.cc:237] T 00000000000000000000000000000000 P fcc64a3087594f0ebdbd89c0c4f82f56 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcc64a3087594f0ebdbd89c0c4f82f56" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 35897 } }
I20260806 09:34:27.138092 20953 sys_catalog.cc:455] T 00000000000000000000000000000000 P fcc64a3087594f0ebdbd89c0c4f82f56 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fcc64a3087594f0ebdbd89c0c4f82f56. Latest consensus state: current_term: 1 leader_uuid: "fcc64a3087594f0ebdbd89c0c4f82f56" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcc64a3087594f0ebdbd89c0c4f82f56" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 35897 } } }
I20260806 09:34:27.139703 20953 sys_catalog.cc:458] T 00000000000000000000000000000000 P fcc64a3087594f0ebdbd89c0c4f82f56 [sys.catalog]: This master's current role is: LEADER
I20260806 09:34:27.139302 20958 sys_catalog.cc:455] T 00000000000000000000000000000000 P fcc64a3087594f0ebdbd89c0c4f82f56 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fcc64a3087594f0ebdbd89c0c4f82f56" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcc64a3087594f0ebdbd89c0c4f82f56" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 35897 } } }
I20260806 09:34:27.140319 20958 sys_catalog.cc:458] T 00000000000000000000000000000000 P fcc64a3087594f0ebdbd89c0c4f82f56 [sys.catalog]: This master's current role is: LEADER
I20260806 09:34:27.142400 20930 sys_catalog.cc:564] T 00000000000000000000000000000000 P fcc64a3087594f0ebdbd89c0c4f82f56 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 09:34:27.142594 20962 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 09:34:27.148996 20962 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 09:34:27.149055 20942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:27.130189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53777 (local address 127.20.113.62:35897)
0806 09:34:27.130376 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0806 09:34:27.130383 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:34:27.130760 (+ 377us) server_negotiation.cc:407] Connection header received
0806 09:34:27.130889 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:34:27.130894 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:34:27.130976 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:34:27.131082 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:34:27.132743 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:27.134191 (+ 1448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:27.135982 (+ 1791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:27.136920 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:27.137083 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:34:27.140531 (+ 3448us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:34:27.140539 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:34:27.140547 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:34:27.140600 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:34:27.145028 (+ 4428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:34:27.145903 (+ 875us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:34:27.145910 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:34:27.145913 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:34:27.146066 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:34:27.146578 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:34:27.146583 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:34:27.146589 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:34:27.147032 (+ 443us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:34:27.147279 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0806 09:34:27.148499 (+ 1220us) server_negotiation.cc:299] Negotiation successful
0806 09:34:27.148709 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260806 09:34:27.160465 20962 catalog_manager.cc:1347] Generated new cluster ID: f6539f15b2784c6c81f4003424c53b77
I20260806 09:34:27.160559 20962 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 09:34:27.179352 20962 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 09:34:27.180601 20962 catalog_manager.cc:1504] Loading token signing keys...
I20260806 09:34:27.188843 20962 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fcc64a3087594f0ebdbd89c0c4f82f56: Generated new TSK 0
I20260806 09:34:27.189594 20962 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 09:34:27.200317 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 20824
I20260806 09:34:27.201444 20921 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:34:27.290881 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 20824
2026-08-06T09:34:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (2353 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786008495763054-20932-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 09:34:27 dist-test-slave-45dx krb5kdc[20989](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 09:34:27 dist-test-slave-45dx krb5kdc[20989](info): set up 2 sockets
Aug 06 09:34:27 dist-test-slave-45dx krb5kdc[20989](info): commencing operation
krb5kdc: starting...
W20260806 09:34:28.594624 20932 mini_kdc.cc:121] Time spent starting KDC: real 1.259s user 0.003s sys 0.055s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786008495763054-20932-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786008495763054-20932-0/krb5kdc/test-user.keytab.
Aug 06 09:34:28 dist-test-slave-45dx krb5kdc[20989](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008868, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T09:34:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T09:34:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.113.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.113.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.113.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab.
Entry for principal kudu/127.20.113.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab.
I20260806 09:34:28.953382 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786008495763054-20932-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786008495763054-20932-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786008495763054-20932-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.113.62:35903
--webserver_interface=127.20.113.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab
--principal=kudu/127.20.113.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:33721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.113.62:35903
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:34:29.296587 21011 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:34:29.296856 21011 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:34:29.297012 21011 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:34:29.304049 21011 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 09:34:29.304152 21011 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:34:29.304198 21011 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:34:29.304291 21011 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 09:34:29.304379 21011 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 09:34:29.313822 21011 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:33721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786008495763054-20932-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786008495763054-20932-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.113.62:35903
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab
--principal=kudu/127.20.113.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.113.62:35903
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.113.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786008495763054-20932-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:34:29.315117 21011 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:34:29.316438 21011 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:34:29.328884 21017 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:34:29.328948 21021 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:34:29.328970 21018 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:34:29.329763 21011 server_base.cc:1034] running on GCE node
I20260806 09:34:29.330546 21011 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:34:29.331904 21011 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:34:29.333132 21011 hybrid_clock.cc:648] HybridClock initialized: now 1786008869333097 us; error 53 us; skew 500 ppm
I20260806 09:34:29.333473 21011 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:34:29 dist-test-slave-45dx krb5kdc[20989](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008869, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:34:29.336648 21011 init.cc:377] Logged in from keytab as kudu/127.20.113.62@KRBTEST.COM (short username kudu)
I20260806 09:34:29.338163 21011 webserver.cc:460] Webserver started at http://127.20.113.62:46863/ using document root <none> and password file <none>
I20260806 09:34:29.338856 21011 fs_manager.cc:359] Metadata directory not provided
I20260806 09:34:29.339023 21011 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:34:29.339325 21011 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:34:29.342424 21011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786008495763054-20932-0/minicluster-data/master-0/data/instance:
uuid: "42538a1f1c83434da332911b480e3d58"
format_stamp: "Formatted at 2026-08-06 09:34:29 on dist-test-slave-45dx"
server_key: "3596d043a185ea4142f848fab800f032"
server_key_iv: "f7a21d77e9298fa09477e5a74117744c"
server_key_version: "encryptionkey@0"
I20260806 09:34:29.343195 21011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786008495763054-20932-0/minicluster-data/master-0/wal/instance:
uuid: "42538a1f1c83434da332911b480e3d58"
format_stamp: "Formatted at 2026-08-06 09:34:29 on dist-test-slave-45dx"
server_key: "3596d043a185ea4142f848fab800f032"
server_key_iv: "f7a21d77e9298fa09477e5a74117744c"
server_key_version: "encryptionkey@0"
I20260806 09:34:29.348488 21011 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260806 09:34:29.351926 21035 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:34:29.353338 21011 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 09:34:29.353528 21011 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786008495763054-20932-0/minicluster-data/master-0/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786008495763054-20932-0/minicluster-data/master-0/wal
uuid: "42538a1f1c83434da332911b480e3d58"
format_stamp: "Formatted at 2026-08-06 09:34:29 on dist-test-slave-45dx"
server_key: "3596d043a185ea4142f848fab800f032"
server_key_iv: "f7a21d77e9298fa09477e5a74117744c"
server_key_version: "encryptionkey@0"
I20260806 09:34:29.353739 21011 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786008495763054-20932-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786008495763054-20932-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786008495763054-20932-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:34:29.370757 21011 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:34:29.374619 21011 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:34:29.374898 21011 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:34:29.393352 21011 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.62:35903
I20260806 09:34:29.393388 21116 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.62:35903 every 8 connection(s)
I20260806 09:34:29.394618 21011 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
I20260806 09:34:29.396727 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 21011
I20260806 09:34:29.396931 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786008495763054-20932-0/minicluster-data/master-0/wal/instance
I20260806 09:34:29.397480 20932 external_mini_cluster.cc:1428] Setting key 1fbcfa698bafc06b68d262d0922ada18
I20260806 09:34:29.398912 21117 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:34:29.410432 21121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:29.399269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37185 (local address 127.20.113.62:35903)
0806 09:34:29.400012 (+ 743us) server_negotiation.cc:206] Beginning negotiation
0806 09:34:29.400023 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 09:34:29.400063 (+ 40us) server_negotiation.cc:407] Connection header received
0806 09:34:29.401670 (+ 1607us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:34:29.401691 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:34:29.402074 (+ 383us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:34:29.402408 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:34:29.402999 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:29.404380 (+ 1381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:29.406231 (+ 1851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:29.406835 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:29.406952 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:34:29.408235 (+ 1283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:34:29.408242 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:34:29.408254 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:34:29.408845 (+ 591us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:34:29.408910 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:34:29.408936 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:34:29.409031 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 09:34:29.409254 (+ 223us) server_negotiation.cc:299] Negotiation successful
0806 09:34:29.409526 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":555,"thread_start_us":351,"threads_started":1}
I20260806 09:34:29.412106 21117 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42538a1f1c83434da332911b480e3d58: Bootstrap starting.
I20260806 09:34:29.414115 20932 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 09:34:29.415426 21117 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 42538a1f1c83434da332911b480e3d58: Neither blocks nor log segments found. Creating new log.
I20260806 09:34:29.416442 21117 log.cc:824] T 00000000000000000000000000000000 P 42538a1f1c83434da332911b480e3d58: Log is configured to *not* fsync() on all Append() calls
I20260806 09:34:29.419152 21117 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42538a1f1c83434da332911b480e3d58: No bootstrap required, opened a new log
I20260806 09:34:29.424916 21121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:29.418439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57931 (local address 127.20.113.62:35903)
0806 09:34:29.418597 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0806 09:34:29.418602 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:34:29.419042 (+ 440us) server_negotiation.cc:407] Connection header received
0806 09:34:29.419178 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:34:29.419183 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:34:29.419253 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:34:29.419351 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:34:29.419861 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:29.420922 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:29.422203 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:29.422902 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:29.423059 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:34:29.424071 (+ 1012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:34:29.424078 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:34:29.424082 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:34:29.424104 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:34:29.424154 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:34:29.424178 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:34:29.424274 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0806 09:34:29.424463 (+ 189us) server_negotiation.cc:299] Negotiation successful
0806 09:34:29.424557 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 09:34:29.427881 21117 raft_consensus.cc:348] T 00000000000000000000000000000000 P 42538a1f1c83434da332911b480e3d58 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42538a1f1c83434da332911b480e3d58" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 35903 } }
I20260806 09:34:29.428225 21117 raft_consensus.cc:374] T 00000000000000000000000000000000 P 42538a1f1c83434da332911b480e3d58 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:34:29.428359 21117 raft_consensus.cc:729] T 00000000000000000000000000000000 P 42538a1f1c83434da332911b480e3d58 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42538a1f1c83434da332911b480e3d58, State: Initialized, Role: FOLLOWER
I20260806 09:34:29.428870 21117 consensus_queue.cc:260] T 00000000000000000000000000000000 P 42538a1f1c83434da332911b480e3d58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42538a1f1c83434da332911b480e3d58" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 35903 } }
I20260806 09:34:29.429121 21117 raft_consensus.cc:388] T 00000000000000000000000000000000 P 42538a1f1c83434da332911b480e3d58 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 09:34:29.429282 21117 raft_consensus.cc:482] T 00000000000000000000000000000000 P 42538a1f1c83434da332911b480e3d58 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 09:34:29.429484 21117 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 42538a1f1c83434da332911b480e3d58 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:34:29.431656 21117 raft_consensus.cc:504] T 00000000000000000000000000000000 P 42538a1f1c83434da332911b480e3d58 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42538a1f1c83434da332911b480e3d58" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 35903 } }
I20260806 09:34:29.432142 21117 leader_election.cc:302] T 00000000000000000000000000000000 P 42538a1f1c83434da332911b480e3d58 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 42538a1f1c83434da332911b480e3d58; no voters:
I20260806 09:34:29.432777 21117 leader_election.cc:288] T 00000000000000000000000000000000 P 42538a1f1c83434da332911b480e3d58 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 09:34:29.432929 21131 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 42538a1f1c83434da332911b480e3d58 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:34:29.433657 21131 raft_consensus.cc:686] T 00000000000000000000000000000000 P 42538a1f1c83434da332911b480e3d58 [term 1 LEADER]: Becoming Leader. State: Replica: 42538a1f1c83434da332911b480e3d58, State: Running, Role: LEADER
I20260806 09:34:29.434227 21131 consensus_queue.cc:237] T 00000000000000000000000000000000 P 42538a1f1c83434da332911b480e3d58 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42538a1f1c83434da332911b480e3d58" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 35903 } }
I20260806 09:34:29.435001 21117 sys_catalog.cc:564] T 00000000000000000000000000000000 P 42538a1f1c83434da332911b480e3d58 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 09:34:29.437392 21134 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42538a1f1c83434da332911b480e3d58 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "42538a1f1c83434da332911b480e3d58" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42538a1f1c83434da332911b480e3d58" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 35903 } } }
I20260806 09:34:29.437414 21135 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42538a1f1c83434da332911b480e3d58 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 42538a1f1c83434da332911b480e3d58. Latest consensus state: current_term: 1 leader_uuid: "42538a1f1c83434da332911b480e3d58" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42538a1f1c83434da332911b480e3d58" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 35903 } } }
I20260806 09:34:29.437813 21135 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42538a1f1c83434da332911b480e3d58 [sys.catalog]: This master's current role is: LEADER
I20260806 09:34:29.437813 21134 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42538a1f1c83434da332911b480e3d58 [sys.catalog]: This master's current role is: LEADER
I20260806 09:34:29.439775 21144 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 09:34:29.446959 21144 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 09:34:29.456293 21144 catalog_manager.cc:1347] Generated new cluster ID: 352fc3985c3446efb455f0b1796e1f32
I20260806 09:34:29.456470 21144 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 09:34:29.482079 21144 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 09:34:29.484107 21144 catalog_manager.cc:1504] Loading token signing keys...
I20260806 09:34:29.494531 21144 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 42538a1f1c83434da332911b480e3d58: Generated new TSK 0
I20260806 09:34:29.495548 21144 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 09:34:29.503907 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 21011
W20260806 09:34:29.505246 21110 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:37185
W20260806 09:34:29.505929 20932 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 21011: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 09:34:29.506089 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 21011
2026-08-06T09:34:29Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (2216 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786008495763054-20932-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 09:34:29 dist-test-slave-45dx krb5kdc[21162](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 09:34:29 dist-test-slave-45dx krb5kdc[21162](info): set up 2 sockets
Aug 06 09:34:29 dist-test-slave-45dx krb5kdc[21162](info): commencing operation
krb5kdc: starting...
W20260806 09:34:30.843652 20932 mini_kdc.cc:121] Time spent starting KDC: real 1.292s user 0.005s sys 0.055s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786008495763054-20932-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786008495763054-20932-0/krb5kdc/test-user.keytab.
Aug 06 09:34:31 dist-test-slave-45dx krb5kdc[21162](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008871, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T09:34:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T09:34:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.113.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.113.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.113.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab.
Entry for principal kudu/127.20.113.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab.
I20260806 09:34:31.221293 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786008495763054-20932-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786008495763054-20932-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786008495763054-20932-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.113.62:38377
--webserver_interface=127.20.113.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab
--principal=kudu/127.20.113.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:35705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.113.62:38377
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:34:31.580523 21181 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:34:31.580940 21181 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:34:31.581106 21181 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:34:31.588749 21181 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 09:34:31.588919 21181 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:34:31.589025 21181 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:34:31.589140 21181 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 09:34:31.589219 21181 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 09:34:31.599377 21181 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:35705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786008495763054-20932-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786008495763054-20932-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.113.62:38377
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab
--principal=kudu/127.20.113.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.113.62:38377
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.113.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786008495763054-20932-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:34:31.600842 21181 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:34:31.602233 21181 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:34:31.614114 21189 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:34:31.614080 21188 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:34:31.614156 21192 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:34:31.615175 21181 server_base.cc:1034] running on GCE node
I20260806 09:34:31.616068 21181 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:34:31.617491 21181 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:34:31.618696 21181 hybrid_clock.cc:648] HybridClock initialized: now 1786008871618654 us; error 33 us; skew 500 ppm
I20260806 09:34:31.619072 21181 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:34:31 dist-test-slave-45dx krb5kdc[21162](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008871, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:34:31.626971 21181 init.cc:377] Logged in from keytab as kudu/127.20.113.62@KRBTEST.COM (short username kudu)
I20260806 09:34:31.628783 21181 webserver.cc:460] Webserver started at http://127.20.113.62:39979/ using document root <none> and password file <none>
I20260806 09:34:31.629571 21181 fs_manager.cc:359] Metadata directory not provided
I20260806 09:34:31.629726 21181 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:34:31.630064 21181 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:34:31.633339 21181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786008495763054-20932-0/minicluster-data/master-0/data/instance:
uuid: "e6599f9c0cc841ff96f248ca03e6d24d"
format_stamp: "Formatted at 2026-08-06 09:34:31 on dist-test-slave-45dx"
server_key: "46d8d12faa3fb24beb2cb9a9ff44c63f"
server_key_iv: "10967731f220f7251ff50ae6298bc0bd"
server_key_version: "encryptionkey@0"
I20260806 09:34:31.634150 21181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786008495763054-20932-0/minicluster-data/master-0/wal/instance:
uuid: "e6599f9c0cc841ff96f248ca03e6d24d"
format_stamp: "Formatted at 2026-08-06 09:34:31 on dist-test-slave-45dx"
server_key: "46d8d12faa3fb24beb2cb9a9ff44c63f"
server_key_iv: "10967731f220f7251ff50ae6298bc0bd"
server_key_version: "encryptionkey@0"
I20260806 09:34:31.639564 21181 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 09:34:31.643097 21204 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:34:31.644475 21181 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 09:34:31.644649 21181 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786008495763054-20932-0/minicluster-data/master-0/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786008495763054-20932-0/minicluster-data/master-0/wal
uuid: "e6599f9c0cc841ff96f248ca03e6d24d"
format_stamp: "Formatted at 2026-08-06 09:34:31 on dist-test-slave-45dx"
server_key: "46d8d12faa3fb24beb2cb9a9ff44c63f"
server_key_iv: "10967731f220f7251ff50ae6298bc0bd"
server_key_version: "encryptionkey@0"
I20260806 09:34:31.644858 21181 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786008495763054-20932-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786008495763054-20932-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786008495763054-20932-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:34:31.650609 21181 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:34:31.654635 21181 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:34:31.654948 21181 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:34:31.673714 21181 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.62:38377
I20260806 09:34:31.673712 21273 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.62:38377 every 8 connection(s)
I20260806 09:34:31.675200 21181 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
I20260806 09:34:31.678531 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 21181
I20260806 09:34:31.678723 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786008495763054-20932-0/minicluster-data/master-0/wal/instance
I20260806 09:34:31.679172 20932 external_mini_cluster.cc:1428] Setting key 6cf2fb0580159861c1069383d56eec15
I20260806 09:34:31.679245 21276 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:34:31.687197 21279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:31.680962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57179 (local address 127.20.113.62:38377)
0806 09:34:31.681782 (+ 820us) server_negotiation.cc:206] Beginning negotiation
0806 09:34:31.681793 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 09:34:31.681839 (+ 46us) server_negotiation.cc:407] Connection header received
0806 09:34:31.683830 (+ 1991us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:34:31.683852 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:34:31.684171 (+ 319us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:34:31.684568 (+ 397us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:34:31.685694 (+ 1126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:34:31.685700 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:34:31.685712 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:34:31.685726 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:34:31.685788 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:34:31.685811 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:34:31.685979 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0806 09:34:31.686082 (+ 103us) server_negotiation.cc:299] Negotiation successful
0806 09:34:31.686279 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":582,"thread_start_us":374,"threads_started":1}
I20260806 09:34:31.690699 20932 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 09:34:31.692893 21276 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6599f9c0cc841ff96f248ca03e6d24d: Bootstrap starting.
I20260806 09:34:31.696906 21276 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e6599f9c0cc841ff96f248ca03e6d24d: Neither blocks nor log segments found. Creating new log.
I20260806 09:34:31.698055 21276 log.cc:824] T 00000000000000000000000000000000 P e6599f9c0cc841ff96f248ca03e6d24d: Log is configured to *not* fsync() on all Append() calls
I20260806 09:34:31.698544 21279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:31.696016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48897 (local address 127.20.113.62:38377)
0806 09:34:31.696192 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0806 09:34:31.696197 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:34:31.696668 (+ 471us) server_negotiation.cc:407] Connection header received
0806 09:34:31.696803 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:34:31.696808 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:34:31.696870 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:34:31.696984 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:34:31.697896 (+ 912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:34:31.697901 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:34:31.697906 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 09:34:31.697922 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:34:31.697965 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 09:34:31.697983 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:34:31.698109 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0806 09:34:31.698184 (+ 75us) server_negotiation.cc:299] Negotiation successful
0806 09:34:31.698241 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 09:34:31.700959 21276 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6599f9c0cc841ff96f248ca03e6d24d: No bootstrap required, opened a new log
I20260806 09:34:31.707993 21276 raft_consensus.cc:348] T 00000000000000000000000000000000 P e6599f9c0cc841ff96f248ca03e6d24d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6599f9c0cc841ff96f248ca03e6d24d" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 38377 } }
I20260806 09:34:31.708367 21276 raft_consensus.cc:374] T 00000000000000000000000000000000 P e6599f9c0cc841ff96f248ca03e6d24d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 09:34:31.708469 21276 raft_consensus.cc:729] T 00000000000000000000000000000000 P e6599f9c0cc841ff96f248ca03e6d24d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6599f9c0cc841ff96f248ca03e6d24d, State: Initialized, Role: FOLLOWER
I20260806 09:34:31.709136 21276 consensus_queue.cc:260] T 00000000000000000000000000000000 P e6599f9c0cc841ff96f248ca03e6d24d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6599f9c0cc841ff96f248ca03e6d24d" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 38377 } }
I20260806 09:34:31.709371 21276 raft_consensus.cc:388] T 00000000000000000000000000000000 P e6599f9c0cc841ff96f248ca03e6d24d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 09:34:31.709517 21276 raft_consensus.cc:482] T 00000000000000000000000000000000 P e6599f9c0cc841ff96f248ca03e6d24d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 09:34:31.709676 21276 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e6599f9c0cc841ff96f248ca03e6d24d [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:34:31.711745 21276 raft_consensus.cc:504] T 00000000000000000000000000000000 P e6599f9c0cc841ff96f248ca03e6d24d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6599f9c0cc841ff96f248ca03e6d24d" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 38377 } }
I20260806 09:34:31.712226 21276 leader_election.cc:302] T 00000000000000000000000000000000 P e6599f9c0cc841ff96f248ca03e6d24d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e6599f9c0cc841ff96f248ca03e6d24d; no voters:
I20260806 09:34:31.712821 21276 leader_election.cc:288] T 00000000000000000000000000000000 P e6599f9c0cc841ff96f248ca03e6d24d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 09:34:31.712951 21289 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e6599f9c0cc841ff96f248ca03e6d24d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:34:31.713639 21289 raft_consensus.cc:686] T 00000000000000000000000000000000 P e6599f9c0cc841ff96f248ca03e6d24d [term 1 LEADER]: Becoming Leader. State: Replica: e6599f9c0cc841ff96f248ca03e6d24d, State: Running, Role: LEADER
I20260806 09:34:31.714283 21289 consensus_queue.cc:237] T 00000000000000000000000000000000 P e6599f9c0cc841ff96f248ca03e6d24d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6599f9c0cc841ff96f248ca03e6d24d" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 38377 } }
I20260806 09:34:31.714841 21276 sys_catalog.cc:564] T 00000000000000000000000000000000 P e6599f9c0cc841ff96f248ca03e6d24d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 09:34:31.717404 21290 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6599f9c0cc841ff96f248ca03e6d24d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e6599f9c0cc841ff96f248ca03e6d24d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6599f9c0cc841ff96f248ca03e6d24d" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 38377 } } }
I20260806 09:34:31.717964 21290 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6599f9c0cc841ff96f248ca03e6d24d [sys.catalog]: This master's current role is: LEADER
I20260806 09:34:31.717774 21292 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6599f9c0cc841ff96f248ca03e6d24d [sys.catalog]: SysCatalogTable state changed. Reason: New leader e6599f9c0cc841ff96f248ca03e6d24d. Latest consensus state: current_term: 1 leader_uuid: "e6599f9c0cc841ff96f248ca03e6d24d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6599f9c0cc841ff96f248ca03e6d24d" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 38377 } } }
I20260806 09:34:31.718102 21292 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6599f9c0cc841ff96f248ca03e6d24d [sys.catalog]: This master's current role is: LEADER
I20260806 09:34:31.719805 21301 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 09:34:31.724895 21301 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 09:34:31.733458 21301 catalog_manager.cc:1347] Generated new cluster ID: f44824c365a545f6b0d4ddb93a077973
I20260806 09:34:31.733568 21301 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 09:34:31.748974 21301 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 09:34:31.749460 21301 catalog_manager.cc:1504] Loading token signing keys...
I20260806 09:34:31.756328 21301 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e6599f9c0cc841ff96f248ca03e6d24d: Generated new TSK 0
I20260806 09:34:31.757045 21301 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 09:34:31.766726 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 21181
W20260806 09:34:31.767850 21269 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:57179
W20260806 09:34:31.768594 20932 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 21181: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 09:34:31.768750 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 21181
2026-08-06T09:34:31Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (2262 ms)
[----------] 18 tests from AuthTokenIssuingTest (41086 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786008495763054-20932-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 09:34:31 dist-test-slave-45dx krb5kdc[21322](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 09:34:31 dist-test-slave-45dx krb5kdc[21322](info): set up 2 sockets
Aug 06 09:34:31 dist-test-slave-45dx krb5kdc[21322](info): commencing operation
krb5kdc: starting...
W20260806 09:34:33.141948 20932 mini_kdc.cc:121] Time spent starting KDC: real 1.327s user 0.000s sys 0.064s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786008495763054-20932-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786008495763054-20932-0/krb5kdc/test-user.keytab.
Aug 06 09:34:33 dist-test-slave-45dx krb5kdc[21322](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008873, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T09:34:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T09:34:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.113.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.113.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.113.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab.
Entry for principal kudu/127.20.113.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab.
I20260806 09:34:33.537993 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786008495763054-20932-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786008495763054-20932-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786008495763054-20932-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.113.62:46435
--webserver_interface=127.20.113.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab
--principal=kudu/127.20.113.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:43651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.113.62:46435,127.20.113.61:43359,127.20.113.60:33399
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:34:33.890559 21342 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 09:34:33.890918 21342 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:34:33.891049 21342 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:34:33.891196 21342 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:34:33.898335 21342 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 09:34:33.898420 21342 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:34:33.898526 21342 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:34:33.898617 21342 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 09:34:33.898706 21342 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 09:34:33.908828 21342 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:43651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786008495763054-20932-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786008495763054-20932-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.113.62:46435,127.20.113.61:43359,127.20.113.60:33399
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab
--principal=kudu/127.20.113.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.113.62:46435
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.113.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786008495763054-20932-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:34:33.910243 21342 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:34:33.911635 21342 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:34:33.923806 21347 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:34:33.923719 21353 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:34:33.923754 21349 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:34:33.924818 21342 server_base.cc:1034] running on GCE node
I20260806 09:34:33.925679 21342 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:34:33.927116 21342 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:34:33.928399 21342 hybrid_clock.cc:648] HybridClock initialized: now 1786008873928318 us; error 45 us; skew 500 ppm
I20260806 09:34:33.928742 21342 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:34:33 dist-test-slave-45dx krb5kdc[21322](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008873, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:34:33.932049 21342 init.cc:377] Logged in from keytab as kudu/127.20.113.62@KRBTEST.COM (short username kudu)
I20260806 09:34:33.933511 21342 webserver.cc:460] Webserver started at http://127.20.113.62:44011/ using document root <none> and password file <none>
I20260806 09:34:33.934214 21342 fs_manager.cc:359] Metadata directory not provided
I20260806 09:34:33.934345 21342 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:34:33.934684 21342 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:34:33.937834 21342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786008495763054-20932-0/minicluster-data/master-0/data/instance:
uuid: "c9017fddf7a94432b842e60d2dba5f93"
format_stamp: "Formatted at 2026-08-06 09:34:33 on dist-test-slave-45dx"
server_key: "2c883b01d3972eb3e68884b497c64954"
server_key_iv: "a458d12e0864a0e857646d4858824b53"
server_key_version: "encryptionkey@0"
I20260806 09:34:33.938658 21342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786008495763054-20932-0/minicluster-data/master-0/wal/instance:
uuid: "c9017fddf7a94432b842e60d2dba5f93"
format_stamp: "Formatted at 2026-08-06 09:34:33 on dist-test-slave-45dx"
server_key: "2c883b01d3972eb3e68884b497c64954"
server_key_iv: "a458d12e0864a0e857646d4858824b53"
server_key_version: "encryptionkey@0"
I20260806 09:34:33.943928 21342 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260806 09:34:33.947489 21363 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:34:33.948822 21342 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 09:34:33.949026 21342 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786008495763054-20932-0/minicluster-data/master-0/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786008495763054-20932-0/minicluster-data/master-0/wal
uuid: "c9017fddf7a94432b842e60d2dba5f93"
format_stamp: "Formatted at 2026-08-06 09:34:33 on dist-test-slave-45dx"
server_key: "2c883b01d3972eb3e68884b497c64954"
server_key_iv: "a458d12e0864a0e857646d4858824b53"
server_key_version: "encryptionkey@0"
I20260806 09:34:33.949261 21342 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786008495763054-20932-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786008495763054-20932-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786008495763054-20932-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:34:33.970448 21342 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:34:33.974858 21342 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:34:33.975149 21342 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:34:33.994333 21342 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.62:46435
I20260806 09:34:33.994374 21441 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.62:46435 every 8 connection(s)
I20260806 09:34:33.995711 21342 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
I20260806 09:34:33.996343 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 21342
I20260806 09:34:33.996615 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786008495763054-20932-0/minicluster-data/master-0/wal/instance
I20260806 09:34:33.997558 20932 external_mini_cluster.cc:1428] Setting key 06a2112bf9bd0499cca2ae9ebdec637e
I20260806 09:34:33.999748 21442 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:34:34.006196 21442 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.113.62" port: 46435 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 09:34:34 dist-test-slave-45dx krb5kdc[21322](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008873, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.62@KRBTEST.COM for kudu/127.20.113.62@KRBTEST.COM
I20260806 09:34:34.031929 21444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:34.009447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:46435 (local address 127.0.0.1:44740)
0806 09:34:34.010444 (+ 997us) negotiation.cc:107] Waiting for socket to connect
0806 09:34:34.010507 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0806 09:34:34.012472 (+ 1965us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:34:34.013441 (+ 969us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:34:34.013446 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:34:34.014010 (+ 564us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:34:34.014490 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:34:34.014513 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:34:34.016942 (+ 2429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:34:34.016946 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:34:34.018219 (+ 1273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:34:34.018238 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:34:34.019581 (+ 1343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:34:34.019586 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:34:34.019752 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:34:34.020497 (+ 745us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:34:34.020526 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:34:34.022916 (+ 2390us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:34:34.027650 (+ 4734us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:34:34.027661 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:34:34.027672 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:34:34.028100 (+ 428us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:34:34.028624 (+ 524us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:34:34.028627 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:34:34.028630 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:34:34.028814 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:34:34.029847 (+ 1033us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:34:34.029853 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:34:34.030054 (+ 201us) client_negotiation.cc:769] Sending connection context
0806 09:34:34.030310 (+ 256us) client_negotiation.cc:240] Negotiation successful
0806 09:34:34.030733 (+ 423us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":637,"mutex_wait_us":1,"thread_start_us":392,"threads_started":1}
I20260806 09:34:34.031960 21445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:34.009445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44740 (local address 127.20.113.62:46435)
0806 09:34:34.011194 (+ 1749us) server_negotiation.cc:206] Beginning negotiation
0806 09:34:34.011205 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 09:34:34.011268 (+ 63us) server_negotiation.cc:407] Connection header received
0806 09:34:34.012869 (+ 1601us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:34:34.012880 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:34:34.013191 (+ 311us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:34:34.013433 (+ 242us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:34:34.014727 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:34.016759 (+ 2032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:34.018402 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:34.019338 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:34.019530 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:34:34.023215 (+ 3685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:34:34.023234 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:34:34.023253 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:34:34.023306 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:34:34.027518 (+ 4212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:34:34.028239 (+ 721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:34:34.028251 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:34:34.028257 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:34:34.028497 (+ 240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:34:34.028935 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:34:34.028943 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:34:34.028948 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:34:34.029702 (+ 754us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:34:34.029911 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0806 09:34:34.030496 (+ 585us) server_negotiation.cc:299] Negotiation successful
0806 09:34:34.030902 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":144,"server-negotiator.queue_time_us":1498,"spinlock_wait_cycles":34048,"thread_start_us":1288,"threads_started":1}
I20260806 09:34:34.035295 21442 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.113.61" port: 43359 } has no permanent_uuid. Determining permanent_uuid...
I20260806 09:34:34.036571 21444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:34.036131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.61:43359 (local address 127.0.0.1:55240)
0806 09:34:34.036338 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0806 09:34:34.036462 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.61:43359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260806 09:34:34.036866 21368 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.113.61:43359: connect: Connection refused (error 111)
I20260806 09:34:34.038583 21444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:34.038177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.61:43359 (local address 127.0.0.1:55248)
0806 09:34:34.038390 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0806 09:34:34.038458 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.61:43359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260806 09:34:34.038801 21442 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.113.61:43359: Network error: Client connection negotiation failed: client connection to 127.20.113.61:43359: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.20.113.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.113.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.113.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.61.keytab.
Entry for principal kudu/127.20.113.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.61.keytab.
I20260806 09:34:34.087622 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786008495763054-20932-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786008495763054-20932-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786008495763054-20932-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786008495763054-20932-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.113.61:43359
--webserver_interface=127.20.113.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.61.keytab
--principal=kudu/127.20.113.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:43651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.113.62:46435,127.20.113.61:43359,127.20.113.60:33399
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 09:34:34.088150 21442 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.113.61" port: 43359 } attempt: 1
I20260806 09:34:34.089579 21444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:34.088938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.61:43359 (local address 127.0.0.1:55262)
0806 09:34:34.089189 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0806 09:34:34.089369 (+ 180us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.61:43359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260806 09:34:34.090747 21444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:34.090325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.61:43359 (local address 127.0.0.1:55274)
0806 09:34:34.090493 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0806 09:34:34.090581 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.61:43359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260806 09:34:34.091033 21442 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.113.61:43359: Network error: Client connection negotiation failed: client connection to 127.20.113.61:43359: connect: Connection refused (error 111)
I20260806 09:34:34.159346 21442 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.113.61" port: 43359 } attempt: 2
I20260806 09:34:34.160771 21444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:34.160094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.61:43359 (local address 127.0.0.1:55276)
0806 09:34:34.160442 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0806 09:34:34.160581 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.61:43359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260806 09:34:34.161823 21444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:34.161421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.61:43359 (local address 127.0.0.1:55282)
0806 09:34:34.161565 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0806 09:34:34.161661 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.61:43359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260806 09:34:34.162040 21442 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.113.61:43359: Network error: Client connection negotiation failed: client connection to 127.20.113.61:43359: connect: Connection refused (error 111)
I20260806 09:34:34.253324 21442 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.113.61" port: 43359 } attempt: 3
I20260806 09:34:34.254722 21444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:34.254166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.61:43359 (local address 127.0.0.1:55294)
0806 09:34:34.254395 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0806 09:34:34.254518 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.61:43359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260806 09:34:34.255901 21444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:34.255477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.61:43359 (local address 127.0.0.1:55304)
0806 09:34:34.255638 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0806 09:34:34.255720 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.61:43359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260806 09:34:34.256135 21442 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.113.61:43359: Network error: Client connection negotiation failed: client connection to 127.20.113.61:43359: connect: Connection refused (error 111)
I20260806 09:34:34.399469 21442 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.113.61" port: 43359 } attempt: 4
I20260806 09:34:34.400880 21444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:34.400209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.61:43359 (local address 127.0.0.1:55312)
0806 09:34:34.400569 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0806 09:34:34.400692 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.61:43359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188}
I20260806 09:34:34.402134 21444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:34.401670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.61:43359 (local address 127.0.0.1:55322)
0806 09:34:34.401861 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0806 09:34:34.401969 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.61:43359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260806 09:34:34.402380 21442 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.113.61:43359: Network error: Client connection negotiation failed: client connection to 127.20.113.61:43359: connect: Connection refused (error 111)
W20260806 09:34:34.457250 21453 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 09:34:34.457610 21453 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:34:34.457691 21453 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:34:34.457772 21453 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:34:34.465215 21453 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 09:34:34.465312 21453 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:34:34.465370 21453 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:34:34.465423 21453 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 09:34:34.465489 21453 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 09:34:34.475467 21453 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:43651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786008495763054-20932-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786008495763054-20932-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.20.113.62:46435,127.20.113.61:43359,127.20.113.60:33399
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.61.keytab
--principal=kudu/127.20.113.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.113.61:43359
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786008495763054-20932-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.20.113.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786008495763054-20932-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:34:34.476709 21453 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:34:34.477986 21453 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:34:34.489676 21461 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:34:34.489737 21459 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:34:34.489737 21464 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:34:34.490614 21453 server_base.cc:1034] running on GCE node
I20260806 09:34:34.491338 21453 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:34:34.492735 21453 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:34:34.493974 21453 hybrid_clock.cc:648] HybridClock initialized: now 1786008874493907 us; error 42 us; skew 500 ppm
I20260806 09:34:34.494303 21453 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:34:34 dist-test-slave-45dx krb5kdc[21322](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008874, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:34:34.497453 21453 init.cc:377] Logged in from keytab as kudu/127.20.113.61@KRBTEST.COM (short username kudu)
I20260806 09:34:34.498723 21453 webserver.cc:460] Webserver started at http://127.20.113.61:44009/ using document root <none> and password file <none>
I20260806 09:34:34.499382 21453 fs_manager.cc:359] Metadata directory not provided
I20260806 09:34:34.499507 21453 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:34:34.499831 21453 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:34:34.502821 21453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786008495763054-20932-0/minicluster-data/master-1/data/instance:
uuid: "04ee6f7d45784380a57983f1bfb82d35"
format_stamp: "Formatted at 2026-08-06 09:34:34 on dist-test-slave-45dx"
server_key: "c05b590d5d4014ea4acd980cd93721f5"
server_key_iv: "adf632e8afe0b84c5bd2dd7f4fbfff7c"
server_key_version: "encryptionkey@0"
I20260806 09:34:34.503634 21453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786008495763054-20932-0/minicluster-data/master-1/wal/instance:
uuid: "04ee6f7d45784380a57983f1bfb82d35"
format_stamp: "Formatted at 2026-08-06 09:34:34 on dist-test-slave-45dx"
server_key: "c05b590d5d4014ea4acd980cd93721f5"
server_key_iv: "adf632e8afe0b84c5bd2dd7f4fbfff7c"
server_key_version: "encryptionkey@0"
I20260806 09:34:34.508661 21453 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260806 09:34:34.512030 21474 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:34:34.513409 21453 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 09:34:34.513592 21453 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786008495763054-20932-0/minicluster-data/master-1/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786008495763054-20932-0/minicluster-data/master-1/wal
uuid: "04ee6f7d45784380a57983f1bfb82d35"
format_stamp: "Formatted at 2026-08-06 09:34:34 on dist-test-slave-45dx"
server_key: "c05b590d5d4014ea4acd980cd93721f5"
server_key_iv: "adf632e8afe0b84c5bd2dd7f4fbfff7c"
server_key_version: "encryptionkey@0"
I20260806 09:34:34.513808 21453 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786008495763054-20932-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786008495763054-20932-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786008495763054-20932-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:34:34.525135 21453 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:34:34.529369 21453 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:34:34.529633 21453 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:34:34.548596 21453 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.61:43359
I20260806 09:34:34.548616 21551 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.61:43359 every 8 connection(s)
I20260806 09:34:34.549929 21453 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786008495763054-20932-0/minicluster-data/master-1/data/info.pb
I20260806 09:34:34.553761 21553 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:34:34.558043 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 21453
I20260806 09:34:34.558236 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786008495763054-20932-0/minicluster-data/master-1/wal/instance
I20260806 09:34:34.558666 20932 external_mini_cluster.cc:1428] Setting key ea717327776a3ec060e7b226f31d0bdf
I20260806 09:34:34.560256 21553 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.113.62" port: 46435 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 09:34:34 dist-test-slave-45dx krb5kdc[21322](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008874, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.61@KRBTEST.COM for kudu/127.20.113.62@KRBTEST.COM
I20260806 09:34:34.579933 21555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:34.562878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44754 (local address 127.20.113.62:46435)
0806 09:34:34.563412 (+ 534us) server_negotiation.cc:206] Beginning negotiation
0806 09:34:34.563417 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:34:34.564263 (+ 846us) server_negotiation.cc:407] Connection header received
0806 09:34:34.566104 (+ 1841us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:34:34.566110 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:34:34.566201 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:34:34.566324 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:34:34.567821 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:34.568627 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:34.570142 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:34.570559 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:34.570690 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:34:34.574158 (+ 3468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:34:34.574162 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:34:34.574165 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:34:34.574200 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:34:34.576846 (+ 2646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:34:34.577645 (+ 799us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:34:34.577650 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:34:34.577652 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:34:34.577778 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:34:34.578262 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:34:34.578265 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:34:34.578267 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:34:34.578565 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:34:34.578677 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 09:34:34.579487 (+ 810us) server_negotiation.cc:299] Negotiation successful
0806 09:34:34.579645 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":430,"thread_start_us":305,"threads_started":1}
I20260806 09:34:34.581481 21557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:34.563373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:46435 (local address 127.0.0.1:44754)
0806 09:34:34.564105 (+ 732us) negotiation.cc:107] Waiting for socket to connect
0806 09:34:34.564142 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 09:34:34.565832 (+ 1690us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:34:34.566549 (+ 717us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:34:34.566557 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:34:34.567078 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:34:34.567611 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:34:34.567628 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:34:34.568791 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:34:34.568813 (+ 22us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:34:34.569990 (+ 1177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:34:34.570004 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:34:34.570783 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:34:34.570791 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:34:34.571009 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:34:34.571698 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:34:34.571719 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:34:34.573946 (+ 2227us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:34:34.577002 (+ 3056us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:34:34.577012 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:34:34.577026 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:34:34.577492 (+ 466us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:34:34.577900 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:34:34.577907 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:34:34.577913 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:34:34.578152 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:34:34.578690 (+ 538us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:34:34.578701 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:34:34.579124 (+ 423us) client_negotiation.cc:769] Sending connection context
0806 09:34:34.579423 (+ 299us) client_negotiation.cc:240] Negotiation successful
0806 09:34:34.579956 (+ 533us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":516,"thread_start_us":316,"threads_started":1}
I20260806 09:34:34.583293 21553 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.113.61" port: 43359 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 09:34:34 dist-test-slave-45dx krb5kdc[21322](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008874, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.61@KRBTEST.COM for kudu/127.20.113.61@KRBTEST.COM
WARNING: no policy specified for kudu/127.20.113.60@KRBTEST.COM; defaulting to no policy
I20260806 09:34:34.599707 21557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:34.584209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.61:43359 (local address 127.0.0.1:55326)
0806 09:34:34.584486 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0806 09:34:34.584507 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 09:34:34.584619 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:34:34.585611 (+ 992us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:34:34.585619 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:34:34.586127 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:34:34.586342 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:34:34.586356 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:34:34.587778 (+ 1422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:34:34.587786 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:34:34.589348 (+ 1562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:34:34.589376 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:34:34.590234 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:34:34.590243 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:34:34.590485 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:34:34.591139 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:34:34.591166 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:34:34.593817 (+ 2651us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:34:34.597191 (+ 3374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:34:34.597199 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:34:34.597204 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:34:34.597567 (+ 363us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:34:34.597969 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:34:34.597976 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:34:34.597980 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:34:34.598135 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:34:34.598708 (+ 573us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:34:34.598712 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:34:34.598852 (+ 140us) client_negotiation.cc:769] Sending connection context
0806 09:34:34.598992 (+ 140us) client_negotiation.cc:240] Negotiation successful
0806 09:34:34.599270 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63}
I20260806 09:34:34.599846 21561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:34.584491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55326 (local address 127.20.113.61:43359)
0806 09:34:34.585181 (+ 690us) server_negotiation.cc:206] Beginning negotiation
0806 09:34:34.585187 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:34:34.585212 (+ 25us) server_negotiation.cc:407] Connection header received
0806 09:34:34.585255 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:34:34.585260 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:34:34.585419 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:34:34.585552 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:34:34.586542 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:34.587629 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:34.589500 (+ 1871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:34.590070 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:34.590168 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:34:34.594038 (+ 3870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:34:34.594050 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:34:34.594064 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:34:34.594092 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:34:34.597065 (+ 2973us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:34:34.597734 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:34:34.597740 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:34:34.597743 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:34:34.597857 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:34:34.598249 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:34:34.598253 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:34:34.598255 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:34:34.598586 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:34:34.598801 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0806 09:34:34.599162 (+ 361us) server_negotiation.cc:299] Negotiation successful
0806 09:34:34.599383 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":551,"thread_start_us":410,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.113.60@KRBTEST.COM" created.
I20260806 09:34:34.602533 21553 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.113.60" port: 33399 } has no permanent_uuid. Determining permanent_uuid...
I20260806 09:34:34.603749 21557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:34.603175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.60:33399 (local address 127.0.0.1:36868)
0806 09:34:34.603376 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0806 09:34:34.603530 (+ 154us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.60:33399: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260806 09:34:34.604033 21476 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.113.60:33399: connect: Connection refused (error 111)
I20260806 09:34:34.605779 21557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:34.605346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.60:33399 (local address 127.0.0.1:36876)
0806 09:34:34.605528 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0806 09:34:34.605634 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.60:33399: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260806 09:34:34.606045 21553 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.113.60:33399: Network error: Client connection negotiation failed: client connection to 127.20.113.60:33399: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.113.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.60.keytab.
Entry for principal kudu/127.20.113.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.60.keytab.
I20260806 09:34:34.646001 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786008495763054-20932-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786008495763054-20932-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786008495763054-20932-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786008495763054-20932-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.113.60:33399
--webserver_interface=127.20.113.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.60.keytab
--principal=kudu/127.20.113.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:43651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.113.62:46435,127.20.113.61:43359,127.20.113.60:33399
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 09:34:34.655357 21553 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.113.60" port: 33399 } attempt: 1
I20260806 09:34:34.656688 21557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:34.656029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.60:33399 (local address 127.0.0.1:36888)
0806 09:34:34.656289 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0806 09:34:34.656468 (+ 179us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.60:33399: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260806 09:34:34.657840 21557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:34.657389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.60:33399 (local address 127.0.0.1:36896)
0806 09:34:34.657576 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0806 09:34:34.657681 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.60:33399: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260806 09:34:34.658106 21553 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.113.60:33399: Network error: Client connection negotiation failed: client connection to 127.20.113.60:33399: connect: Connection refused (error 111)
I20260806 09:34:34.701661 21442 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.113.61" port: 43359 } attempt: 5
Aug 06 09:34:34 dist-test-slave-45dx krb5kdc[21322](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008873, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.62@KRBTEST.COM for kudu/127.20.113.61@KRBTEST.COM
I20260806 09:34:34.716480 21444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:34.702463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.61:43359 (local address 127.0.0.1:55340)
0806 09:34:34.702699 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0806 09:34:34.702721 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 09:34:34.702883 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:34:34.703280 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:34:34.703285 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:34:34.703696 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:34:34.703902 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:34:34.703911 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:34:34.704994 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:34:34.705001 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:34:34.705884 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:34:34.705895 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:34:34.706667 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:34:34.706672 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:34:34.706891 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:34:34.707555 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:34:34.707574 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:34:34.709725 (+ 2151us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:34:34.713791 (+ 4066us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:34:34.713796 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:34:34.713799 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:34:34.714196 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:34:34.714671 (+ 475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:34:34.714674 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:34:34.714676 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:34:34.714829 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:34:34.715616 (+ 787us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:34:34.715620 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:34:34.715788 (+ 168us) client_negotiation.cc:769] Sending connection context
0806 09:34:34.715907 (+ 119us) client_negotiation.cc:240] Negotiation successful
0806 09:34:34.716129 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":79}
I20260806 09:34:34.717068 21561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:34.702573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55340 (local address 127.20.113.61:43359)
0806 09:34:34.702764 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0806 09:34:34.702771 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:34:34.702794 (+ 23us) server_negotiation.cc:407] Connection header received
0806 09:34:34.703038 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:34:34.703043 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:34:34.703127 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:34:34.703233 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:34:34.704064 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:34.704857 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:34.706059 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:34.706518 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:34.706654 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:34:34.709943 (+ 3289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:34:34.709951 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:34:34.709958 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:34:34.710000 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:34:34.713630 (+ 3630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:34:34.714326 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:34:34.714335 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:34:34.714340 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:34:34.714529 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:34:34.714950 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:34:34.714982 (+ 32us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:34:34.714987 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:34:34.715458 (+ 471us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:34:34.715632 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0806 09:34:34.716075 (+ 443us) server_negotiation.cc:299] Negotiation successful
0806 09:34:34.716414 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260806 09:34:34.718129 21442 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.113.60" port: 33399 } has no permanent_uuid. Determining permanent_uuid...
I20260806 09:34:34.719388 21444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:34.718892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.60:33399 (local address 127.0.0.1:36906)
0806 09:34:34.719141 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0806 09:34:34.719228 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.60:33399: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125}
I20260806 09:34:34.720407 21444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:34.720011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.60:33399 (local address 127.0.0.1:36916)
0806 09:34:34.720158 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0806 09:34:34.720241 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.60:33399: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260806 09:34:34.720593 21442 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.113.60:33399: Network error: Client connection negotiation failed: client connection to 127.20.113.60:33399: connect: Connection refused (error 111)
I20260806 09:34:34.726341 21553 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.113.60" port: 33399 } attempt: 2
I20260806 09:34:34.727195 21557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:34.726830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.60:33399 (local address 127.0.0.1:36924)
0806 09:34:34.726985 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0806 09:34:34.727065 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.60:33399: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260806 09:34:34.728209 21557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:34.727796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.60:33399 (local address 127.0.0.1:36928)
0806 09:34:34.727952 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0806 09:34:34.728042 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.60:33399: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260806 09:34:34.728459 21553 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.113.60:33399: Network error: Client connection negotiation failed: client connection to 127.20.113.60:33399: connect: Connection refused (error 111)
I20260806 09:34:34.771836 21442 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.113.60" port: 33399 } attempt: 1
I20260806 09:34:34.773033 21444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:34.772550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.60:33399 (local address 127.0.0.1:36940)
0806 09:34:34.772758 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0806 09:34:34.772863 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.60:33399: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260806 09:34:34.774142 21444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:34.773830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.60:33399 (local address 127.0.0.1:36948)
0806 09:34:34.773968 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0806 09:34:34.774034 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.60:33399: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260806 09:34:34.774376 21442 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.113.60:33399: Network error: Client connection negotiation failed: client connection to 127.20.113.60:33399: connect: Connection refused (error 111)
I20260806 09:34:34.819701 21553 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.113.60" port: 33399 } attempt: 3
I20260806 09:34:34.821043 21557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:34.820467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.60:33399 (local address 127.0.0.1:36958)
0806 09:34:34.820711 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0806 09:34:34.820860 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.60:33399: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260806 09:34:34.822297 21557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:34.821854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.60:33399 (local address 127.0.0.1:36972)
0806 09:34:34.822089 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0806 09:34:34.822178 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.60:33399: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126}
W20260806 09:34:34.822592 21553 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.113.60:33399: Network error: Client connection negotiation failed: client connection to 127.20.113.60:33399: connect: Connection refused (error 111)
I20260806 09:34:34.842586 21442 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.113.60" port: 33399 } attempt: 2
I20260806 09:34:34.843842 21444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:34.843336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.60:33399 (local address 127.0.0.1:36982)
0806 09:34:34.843576 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0806 09:34:34.843674 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.60:33399: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260806 09:34:34.844847 21444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:34.844508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.60:33399 (local address 127.0.0.1:36998)
0806 09:34:34.844652 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0806 09:34:34.844725 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.60:33399: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260806 09:34:34.845057 21442 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.113.60:33399: Network error: Client connection negotiation failed: client connection to 127.20.113.60:33399: connect: Connection refused (error 111)
I20260806 09:34:34.951267 21442 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.113.60" port: 33399 } attempt: 3
I20260806 09:34:34.952461 21444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:34.951925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.60:33399 (local address 127.0.0.1:37006)
0806 09:34:34.952133 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0806 09:34:34.952227 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.60:33399: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260806 09:34:34.953533 21444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:34.953137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.60:33399 (local address 127.0.0.1:37020)
0806 09:34:34.953355 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0806 09:34:34.953427 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.60:33399: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
W20260806 09:34:34.953775 21442 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.113.60:33399: Network error: Client connection negotiation failed: client connection to 127.20.113.60:33399: connect: Connection refused (error 111)
I20260806 09:34:34.965855 21553 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.113.60" port: 33399 } attempt: 4
I20260806 09:34:34.966952 21557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:34.966494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.60:33399 (local address 127.0.0.1:37028)
0806 09:34:34.966711 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0806 09:34:34.966787 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.60:33399: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260806 09:34:34.968057 21557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:34.967648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.60:33399 (local address 127.0.0.1:37034)
0806 09:34:34.967824 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0806 09:34:34.967921 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.60:33399: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260806 09:34:34.968348 21553 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.113.60:33399: Network error: Client connection negotiation failed: client connection to 127.20.113.60:33399: connect: Connection refused (error 111)
W20260806 09:34:35.012768 21564 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 09:34:35.013082 21564 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:34:35.013160 21564 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:34:35.013242 21564 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:34:35.020707 21564 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 09:34:35.020798 21564 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:34:35.020855 21564 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:34:35.020916 21564 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 09:34:35.021005 21564 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 09:34:35.030854 21564 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:43651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786008495763054-20932-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786008495763054-20932-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.20.113.62:46435,127.20.113.61:43359,127.20.113.60:33399
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.60.keytab
--principal=kudu/127.20.113.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.113.60:33399
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786008495763054-20932-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.20.113.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786008495763054-20932-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:34:35.032030 21564 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:34:35.033377 21564 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:34:35.045310 21571 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:34:35.045261 21576 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:34:35.045332 21572 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:34:35.046572 21564 server_base.cc:1034] running on GCE node
I20260806 09:34:35.047364 21564 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:34:35.048833 21564 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:34:35.050089 21564 hybrid_clock.cc:648] HybridClock initialized: now 1786008875050024 us; error 40 us; skew 500 ppm
I20260806 09:34:35.050418 21564 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:34:35 dist-test-slave-45dx krb5kdc[21322](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008875, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:34:35.053790 21564 init.cc:377] Logged in from keytab as kudu/127.20.113.60@KRBTEST.COM (short username kudu)
I20260806 09:34:35.055279 21564 webserver.cc:460] Webserver started at http://127.20.113.60:33159/ using document root <none> and password file <none>
I20260806 09:34:35.055967 21564 fs_manager.cc:359] Metadata directory not provided
I20260806 09:34:35.056097 21564 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:34:35.056475 21564 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:34:35.059443 21564 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786008495763054-20932-0/minicluster-data/master-2/data/instance:
uuid: "4f5cec1c22924a989447fcd90ff95d77"
format_stamp: "Formatted at 2026-08-06 09:34:35 on dist-test-slave-45dx"
server_key: "957724f80073aaba9dab60a559506ad2"
server_key_iv: "1aed8dc22016bf90192d6cc1c9455880"
server_key_version: "encryptionkey@0"
I20260806 09:34:35.060235 21564 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786008495763054-20932-0/minicluster-data/master-2/wal/instance:
uuid: "4f5cec1c22924a989447fcd90ff95d77"
format_stamp: "Formatted at 2026-08-06 09:34:35 on dist-test-slave-45dx"
server_key: "957724f80073aaba9dab60a559506ad2"
server_key_iv: "1aed8dc22016bf90192d6cc1c9455880"
server_key_version: "encryptionkey@0"
I20260806 09:34:35.065291 21564 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260806 09:34:35.068701 21587 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:34:35.070062 21564 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 09:34:35.070233 21564 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786008495763054-20932-0/minicluster-data/master-2/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786008495763054-20932-0/minicluster-data/master-2/wal
uuid: "4f5cec1c22924a989447fcd90ff95d77"
format_stamp: "Formatted at 2026-08-06 09:34:35 on dist-test-slave-45dx"
server_key: "957724f80073aaba9dab60a559506ad2"
server_key_iv: "1aed8dc22016bf90192d6cc1c9455880"
server_key_version: "encryptionkey@0"
I20260806 09:34:35.070459 21564 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786008495763054-20932-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786008495763054-20932-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786008495763054-20932-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:34:35.131011 21442 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.113.60" port: 33399 } attempt: 4
I20260806 09:34:35.132164 21444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:35.131714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.60:33399 (local address 127.0.0.1:37044)
0806 09:34:35.131919 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0806 09:34:35.132018 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.60:33399: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260806 09:34:35.132514 21564 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:34:35.133239 21444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:35.132851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.60:33399 (local address 127.0.0.1:37052)
0806 09:34:35.133001 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0806 09:34:35.133071 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.60:33399: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260806 09:34:35.133495 21442 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.113.60:33399: Network error: Client connection negotiation failed: client connection to 127.20.113.60:33399: connect: Connection refused (error 111)
I20260806 09:34:35.137101 21564 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:34:35.137388 21564 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:34:35.157523 21564 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.60:33399
I20260806 09:34:35.157575 21662 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.60:33399 every 8 connection(s)
I20260806 09:34:35.158953 21564 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786008495763054-20932-0/minicluster-data/master-2/data/info.pb
I20260806 09:34:35.163053 21664 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:34:35.166543 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 21564
I20260806 09:34:35.166736 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786008495763054-20932-0/minicluster-data/master-2/wal/instance
I20260806 09:34:35.167253 20932 external_mini_cluster.cc:1428] Setting key bf5d0ed22a598090b7814a8f737a40f8
I20260806 09:34:35.172626 21664 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.113.62" port: 46435 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 09:34:35 dist-test-slave-45dx krb5kdc[21322](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008873, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.113.60@KRBTEST.COM
Aug 06 09:34:35 dist-test-slave-45dx krb5kdc[21322](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008875, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.60@KRBTEST.COM for kudu/127.20.113.62@KRBTEST.COM
I20260806 09:34:35.192214 21672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:35.175824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44764 (local address 127.20.113.62:46435)
0806 09:34:35.177356 (+ 1532us) server_negotiation.cc:206] Beginning negotiation
0806 09:34:35.177363 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:34:35.177383 (+ 20us) server_negotiation.cc:407] Connection header received
0806 09:34:35.177445 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:34:35.177451 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:34:35.177541 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:34:35.177656 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:34:35.179158 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:35.180176 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:35.182259 (+ 2083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:35.182676 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:35.182794 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:34:35.186674 (+ 3880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:34:35.186679 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:34:35.186682 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:34:35.186716 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:34:35.189364 (+ 2648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:34:35.190055 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:34:35.190060 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:34:35.190062 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:34:35.190190 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:34:35.190586 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:34:35.190590 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:34:35.190591 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:34:35.190834 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:34:35.191002 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0806 09:34:35.191531 (+ 529us) server_negotiation.cc:299] Negotiation successful
0806 09:34:35.191749 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1378,"thread_start_us":1215,"threads_started":1}
I20260806 09:34:35.193250 21671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:35.175759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:46435 (local address 127.0.0.1:44764)
0806 09:34:35.176547 (+ 788us) negotiation.cc:107] Waiting for socket to connect
0806 09:34:35.176611 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0806 09:34:35.176816 (+ 205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:34:35.177725 (+ 909us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:34:35.177730 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:34:35.178590 (+ 860us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:34:35.178920 (+ 330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:34:35.178934 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:34:35.180448 (+ 1514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:34:35.180453 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:34:35.182026 (+ 1573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:34:35.182048 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:34:35.182956 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:34:35.182966 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:34:35.183186 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:34:35.183859 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:34:35.183892 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:34:35.186461 (+ 2569us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:34:35.189504 (+ 3043us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:34:35.189518 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:34:35.189530 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:34:35.189926 (+ 396us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:34:35.190303 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:34:35.190308 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:34:35.190311 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:34:35.190467 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:34:35.190967 (+ 500us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:34:35.190975 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:34:35.191188 (+ 213us) client_negotiation.cc:769] Sending connection context
0806 09:34:35.191434 (+ 246us) client_negotiation.cc:240] Negotiation successful
0806 09:34:35.191981 (+ 547us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":334,"threads_started":1}
I20260806 09:34:35.193338 21670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:35.169210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37066 (local address 127.20.113.60:33399)
0806 09:34:35.170119 (+ 909us) server_negotiation.cc:206] Beginning negotiation
0806 09:34:35.170137 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 09:34:35.170190 (+ 53us) server_negotiation.cc:407] Connection header received
0806 09:34:35.172889 (+ 2699us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:34:35.172924 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:34:35.173493 (+ 569us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:34:35.173987 (+ 494us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:34:35.175630 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:35.177923 (+ 2293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:35.179642 (+ 1719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:35.180678 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:35.180823 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:34:35.186112 (+ 5289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:34:35.186122 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:34:35.186138 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:34:35.186176 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:34:35.189325 (+ 3149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:34:35.190011 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:34:35.190024 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:34:35.190026 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:34:35.190168 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:34:35.190541 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:34:35.190546 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:34:35.190549 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:34:35.191054 (+ 505us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:34:35.191369 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0806 09:34:35.191714 (+ 345us) server_negotiation.cc:299] Negotiation successful
0806 09:34:35.192157 (+ 443us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":122,"server-negotiator.queue_time_us":645,"thread_start_us":345,"threads_started":1}
I20260806 09:34:35.195719 21664 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.113.61" port: 43359 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 09:34:35 dist-test-slave-45dx krb5kdc[21322](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008875, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.60@KRBTEST.COM for kudu/127.20.113.61@KRBTEST.COM
Aug 06 09:34:35 dist-test-slave-45dx krb5kdc[21322](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008873, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.113.62@KRBTEST.COM
I20260806 09:34:35.210556 21671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:35.196714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.61:43359 (local address 127.0.0.1:55342)
0806 09:34:35.196870 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0806 09:34:35.196887 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 09:34:35.196981 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:34:35.197245 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:34:35.197250 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:34:35.197671 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:34:35.197805 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:34:35.197812 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:34:35.199069 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:34:35.199074 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:34:35.200211 (+ 1137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:34:35.200222 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:34:35.201202 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:34:35.201206 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:34:35.201360 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:34:35.201976 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:34:35.202000 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:34:35.204841 (+ 2841us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:34:35.208075 (+ 3234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:34:35.208079 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:34:35.208082 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:34:35.208506 (+ 424us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:34:35.208900 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:34:35.208904 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:34:35.208906 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:34:35.209058 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:34:35.209588 (+ 530us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:34:35.209593 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:34:35.209747 (+ 154us) client_negotiation.cc:769] Sending connection context
0806 09:34:35.209893 (+ 146us) client_negotiation.cc:240] Negotiation successful
0806 09:34:35.210170 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260806 09:34:35.210592 21561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:35.196671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55342 (local address 127.20.113.61:43359)
0806 09:34:35.196896 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0806 09:34:35.196902 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:34:35.196946 (+ 44us) server_negotiation.cc:407] Connection header received
0806 09:34:35.197057 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:34:35.197061 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:34:35.197120 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:34:35.197194 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:34:35.197959 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:35.198940 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:35.200374 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:35.201065 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:35.201203 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:34:35.205026 (+ 3823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:34:35.205031 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:34:35.205034 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:34:35.205068 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:34:35.207938 (+ 2870us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:34:35.208619 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:34:35.208625 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:34:35.208629 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:34:35.208773 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:34:35.209169 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:34:35.209175 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:34:35.209178 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:34:35.209459 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:34:35.209628 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0806 09:34:35.209990 (+ 362us) server_negotiation.cc:299] Negotiation successful
0806 09:34:35.210201 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260806 09:34:35.211653 21664 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.113.60" port: 33399 } has no permanent_uuid. Determining permanent_uuid...
I20260806 09:34:35.212365 21672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:35.196510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44766 (local address 127.20.113.62:46435)
0806 09:34:35.197152 (+ 642us) server_negotiation.cc:206] Beginning negotiation
0806 09:34:35.197155 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 09:34:35.197190 (+ 35us) server_negotiation.cc:407] Connection header received
0806 09:34:35.197237 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:34:35.197240 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:34:35.197319 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:34:35.197419 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:34:35.198784 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:35.199771 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:35.201411 (+ 1640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:35.202007 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:35.202145 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:34:35.206676 (+ 4531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:34:35.206680 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:34:35.206683 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:34:35.206714 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:34:35.209399 (+ 2685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:34:35.210056 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:34:35.210060 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:34:35.210062 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:34:35.210181 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:34:35.210564 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:34:35.210569 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:34:35.210576 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:34:35.211137 (+ 561us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:34:35.211434 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0806 09:34:35.211731 (+ 297us) server_negotiation.cc:299] Negotiation successful
0806 09:34:35.211984 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":541}
Aug 06 09:34:35 dist-test-slave-45dx krb5kdc[21322](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008875, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.60@KRBTEST.COM for kudu/127.20.113.60@KRBTEST.COM
Aug 06 09:34:35 dist-test-slave-45dx krb5kdc[21322](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008873, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.113.61@KRBTEST.COM
I20260806 09:34:35.227216 21671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:35.212642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.60:33399 (local address 127.0.0.1:37072)
0806 09:34:35.212791 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0806 09:34:35.212811 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 09:34:35.212921 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:34:35.213244 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:34:35.213248 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:34:35.213651 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:34:35.213824 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:34:35.213835 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:34:35.215107 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:34:35.215113 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:34:35.216519 (+ 1406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:34:35.216533 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:34:35.217464 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:34:35.217468 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:34:35.217778 (+ 310us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:34:35.218353 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:34:35.218375 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:34:35.221008 (+ 2633us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:34:35.224167 (+ 3159us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:34:35.224173 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:34:35.224176 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:34:35.224687 (+ 511us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:34:35.225152 (+ 465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:34:35.225161 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:34:35.225168 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:34:35.225362 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:34:35.226011 (+ 649us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:34:35.226027 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:34:35.226204 (+ 177us) client_negotiation.cc:769] Sending connection context
0806 09:34:35.226445 (+ 241us) client_negotiation.cc:240] Negotiation successful
0806 09:34:35.226675 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39,"mutex_wait_us":1}
I20260806 09:34:35.227249 21670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:35.212653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37072 (local address 127.20.113.60:33399)
0806 09:34:35.212828 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0806 09:34:35.212835 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:34:35.212856 (+ 21us) server_negotiation.cc:407] Connection header received
0806 09:34:35.213048 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:34:35.213052 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:34:35.213107 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:34:35.213187 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:34:35.213975 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:35.214977 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:35.216668 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:35.217321 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:35.217461 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:34:35.221210 (+ 3749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:34:35.221215 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:34:35.221218 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:34:35.221248 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:34:35.224025 (+ 2777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:34:35.224842 (+ 817us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:34:35.224851 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:34:35.224856 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:34:35.225026 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:34:35.225488 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:34:35.225493 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:34:35.225497 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:34:35.225857 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:34:35.226058 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0806 09:34:35.226585 (+ 527us) server_negotiation.cc:299] Negotiation successful
0806 09:34:35.226833 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":53,"server-negotiator.queue_time_us":72}
I20260806 09:34:35.230772 21561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:35.214421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55354 (local address 127.20.113.61:43359)
0806 09:34:35.214610 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0806 09:34:35.214616 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:34:35.214632 (+ 16us) server_negotiation.cc:407] Connection header received
0806 09:34:35.214679 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:34:35.214684 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:34:35.214745 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:34:35.214851 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:34:35.216386 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:35.217442 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:35.219239 (+ 1797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:35.219869 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:35.220029 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:34:35.224465 (+ 4436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:34:35.224470 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:34:35.224473 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:34:35.224502 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:34:35.228060 (+ 3558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:34:35.228801 (+ 741us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:34:35.228807 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:34:35.228810 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:34:35.228985 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:34:35.229347 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:34:35.229355 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:34:35.229361 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:34:35.229739 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:34:35.230029 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0806 09:34:35.230185 (+ 156us) server_negotiation.cc:299] Negotiation successful
0806 09:34:35.230414 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 09:34:35.233693 21664 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f5cec1c22924a989447fcd90ff95d77: Bootstrap starting.
I20260806 09:34:35.235841 20932 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 09:34:35.237582 21664 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4f5cec1c22924a989447fcd90ff95d77: Neither blocks nor log segments found. Creating new log.
I20260806 09:34:35.238961 21664 log.cc:824] T 00000000000000000000000000000000 P 4f5cec1c22924a989447fcd90ff95d77: Log is configured to *not* fsync() on all Append() calls
I20260806 09:34:35.242002 21664 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f5cec1c22924a989447fcd90ff95d77: No bootstrap required, opened a new log
I20260806 09:34:35.239323 21405 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "c9017fddf7a94432b842e60d2dba5f93"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44766
I20260806 09:34:35.249145 21664 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4f5cec1c22924a989447fcd90ff95d77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9017fddf7a94432b842e60d2dba5f93" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 46435 } } peers { permanent_uuid: "04ee6f7d45784380a57983f1bfb82d35" member_type: VOTER last_known_addr { host: "127.20.113.61" port: 43359 } } peers { permanent_uuid: "4f5cec1c22924a989447fcd90ff95d77" member_type: VOTER last_known_addr { host: "127.20.113.60" port: 33399 } }
I20260806 09:34:35.249505 21664 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4f5cec1c22924a989447fcd90ff95d77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f5cec1c22924a989447fcd90ff95d77, State: Initialized, Role: FOLLOWER
I20260806 09:34:35.249924 21664 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4f5cec1c22924a989447fcd90ff95d77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9017fddf7a94432b842e60d2dba5f93" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 46435 } } peers { permanent_uuid: "04ee6f7d45784380a57983f1bfb82d35" member_type: VOTER last_known_addr { host: "127.20.113.61" port: 43359 } } peers { permanent_uuid: "4f5cec1c22924a989447fcd90ff95d77" member_type: VOTER last_known_addr { host: "127.20.113.60" port: 33399 } }
I20260806 09:34:35.250927 21675 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f5cec1c22924a989447fcd90ff95d77 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9017fddf7a94432b842e60d2dba5f93" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 46435 } } peers { permanent_uuid: "04ee6f7d45784380a57983f1bfb82d35" member_type: VOTER last_known_addr { host: "127.20.113.61" port: 43359 } } peers { permanent_uuid: "4f5cec1c22924a989447fcd90ff95d77" member_type: VOTER last_known_addr { host: "127.20.113.60" port: 33399 } } }
I20260806 09:34:35.251331 21675 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f5cec1c22924a989447fcd90ff95d77 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 09:34:35.251856 21664 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4f5cec1c22924a989447fcd90ff95d77 [sys.catalog]: configured and running, proceeding with master startup.
W20260806 09:34:35.262009 21689 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4f5cec1c22924a989447fcd90ff95d77: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 09:34:35.262197 21689 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 09:34:35.267635 21553 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.113.60" port: 33399 } attempt: 5
I20260806 09:34:35.260740 21564 master_runner.cc:186] Error getting master registration for 127.20.113.62:46435: OK, instance_id { permanent_uuid: "c9017fddf7a94432b842e60d2dba5f93" instance_seqno: 1786008873988920 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 09:34:35.273259 21564 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Aug 06 09:34:35 dist-test-slave-45dx krb5kdc[21322](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008874, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.61@KRBTEST.COM for kudu/127.20.113.60@KRBTEST.COM
I20260806 09:34:35.285871 21557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:35.268440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.60:33399 (local address 127.0.0.1:37086)
0806 09:34:35.268738 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0806 09:34:35.268762 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 09:34:35.268965 (+ 203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:34:35.269376 (+ 411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:34:35.269382 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:34:35.269893 (+ 511us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:34:35.270068 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:34:35.270078 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:34:35.271817 (+ 1739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:34:35.271823 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:34:35.273470 (+ 1647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:34:35.273487 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:34:35.274577 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:34:35.274583 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:34:35.274836 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:34:35.275537 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:34:35.275561 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:34:35.278548 (+ 2987us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:34:35.282588 (+ 4040us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:34:35.282595 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:34:35.282603 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:34:35.283219 (+ 616us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:34:35.283648 (+ 429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:34:35.283653 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:34:35.283657 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:34:35.283849 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:34:35.284616 (+ 767us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:34:35.284621 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:34:35.284890 (+ 269us) client_negotiation.cc:769] Sending connection context
0806 09:34:35.285062 (+ 172us) client_negotiation.cc:240] Negotiation successful
0806 09:34:35.285405 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177}
I20260806 09:34:35.285884 21670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:35.268516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37086 (local address 127.20.113.60:33399)
0806 09:34:35.268731 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0806 09:34:35.268738 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:34:35.268886 (+ 148us) server_negotiation.cc:407] Connection header received
0806 09:34:35.269112 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:34:35.269118 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:34:35.269214 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:34:35.269337 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:34:35.270255 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:35.271598 (+ 1343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:35.273636 (+ 2038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:35.274397 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:35.274545 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:34:35.278744 (+ 4199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:34:35.278755 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:34:35.278759 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:34:35.278800 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:34:35.282447 (+ 3647us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:34:35.283361 (+ 914us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:34:35.283366 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:34:35.283369 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:34:35.283522 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:34:35.283980 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:34:35.283985 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:34:35.283988 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:34:35.284466 (+ 478us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:34:35.284617 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0806 09:34:35.285272 (+ 655us) server_negotiation.cc:299] Negotiation successful
0806 09:34:35.285502 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260806 09:34:35.290774 21553 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04ee6f7d45784380a57983f1bfb82d35: Bootstrap starting.
I20260806 09:34:35.295224 21553 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 04ee6f7d45784380a57983f1bfb82d35: Neither blocks nor log segments found. Creating new log.
I20260806 09:34:35.296679 21553 log.cc:824] T 00000000000000000000000000000000 P 04ee6f7d45784380a57983f1bfb82d35: Log is configured to *not* fsync() on all Append() calls
I20260806 09:34:35.299911 21553 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04ee6f7d45784380a57983f1bfb82d35: No bootstrap required, opened a new log
I20260806 09:34:35.312696 21553 raft_consensus.cc:348] T 00000000000000000000000000000000 P 04ee6f7d45784380a57983f1bfb82d35 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9017fddf7a94432b842e60d2dba5f93" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 46435 } } peers { permanent_uuid: "04ee6f7d45784380a57983f1bfb82d35" member_type: VOTER last_known_addr { host: "127.20.113.61" port: 43359 } } peers { permanent_uuid: "4f5cec1c22924a989447fcd90ff95d77" member_type: VOTER last_known_addr { host: "127.20.113.60" port: 33399 } }
I20260806 09:34:35.313220 21553 raft_consensus.cc:729] T 00000000000000000000000000000000 P 04ee6f7d45784380a57983f1bfb82d35 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04ee6f7d45784380a57983f1bfb82d35, State: Initialized, Role: FOLLOWER
I20260806 09:34:35.313830 21553 consensus_queue.cc:260] T 00000000000000000000000000000000 P 04ee6f7d45784380a57983f1bfb82d35 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9017fddf7a94432b842e60d2dba5f93" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 46435 } } peers { permanent_uuid: "04ee6f7d45784380a57983f1bfb82d35" member_type: VOTER last_known_addr { host: "127.20.113.61" port: 43359 } } peers { permanent_uuid: "4f5cec1c22924a989447fcd90ff95d77" member_type: VOTER last_known_addr { host: "127.20.113.60" port: 33399 } }
I20260806 09:34:35.314993 21694 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04ee6f7d45784380a57983f1bfb82d35 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9017fddf7a94432b842e60d2dba5f93" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 46435 } } peers { permanent_uuid: "04ee6f7d45784380a57983f1bfb82d35" member_type: VOTER last_known_addr { host: "127.20.113.61" port: 43359 } } peers { permanent_uuid: "4f5cec1c22924a989447fcd90ff95d77" member_type: VOTER last_known_addr { host: "127.20.113.60" port: 33399 } } }
I20260806 09:34:35.315506 21694 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04ee6f7d45784380a57983f1bfb82d35 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 09:34:35.316383 21553 sys_catalog.cc:564] T 00000000000000000000000000000000 P 04ee6f7d45784380a57983f1bfb82d35 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 09:34:35.324182 21453 master_runner.cc:186] Error getting master registration for 127.20.113.62:46435: OK, instance_id { permanent_uuid: "c9017fddf7a94432b842e60d2dba5f93" instance_seqno: 1786008873988920 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 09:34:35.335602 21453 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260806 09:34:35.338941 21710 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 04ee6f7d45784380a57983f1bfb82d35: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 09:34:35.339085 21710 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260806 09:34:35.389719 21599 tablet.cc:2369] T 00000000000000000000000000000000 P 4f5cec1c22924a989447fcd90ff95d77: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 09:34:35.410744 21442 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.113.60" port: 33399 } attempt: 5
Aug 06 09:34:35 dist-test-slave-45dx krb5kdc[21322](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008873, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.62@KRBTEST.COM for kudu/127.20.113.60@KRBTEST.COM
I20260806 09:34:35.428262 21444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:35.411555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.60:33399 (local address 127.0.0.1:37088)
0806 09:34:35.411832 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0806 09:34:35.411854 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 09:34:35.412011 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:34:35.412514 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:34:35.412519 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:34:35.413098 (+ 579us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:34:35.413363 (+ 265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:34:35.413375 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:34:35.414490 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:34:35.414497 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:34:35.416135 (+ 1638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:34:35.416150 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:34:35.417085 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:34:35.417095 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:34:35.417384 (+ 289us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:34:35.418209 (+ 825us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:34:35.418234 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:34:35.421384 (+ 3150us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:34:35.425291 (+ 3907us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:34:35.425300 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:34:35.425305 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:34:35.425824 (+ 519us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:34:35.426228 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:34:35.426233 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:34:35.426236 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:34:35.426445 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:34:35.427183 (+ 738us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:34:35.427188 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:34:35.427346 (+ 158us) client_negotiation.cc:769] Sending connection context
0806 09:34:35.427494 (+ 148us) client_negotiation.cc:240] Negotiation successful
0806 09:34:35.427794 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171}
I20260806 09:34:35.428247 21670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:35.411624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37088 (local address 127.20.113.60:33399)
0806 09:34:35.411824 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0806 09:34:35.411830 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:34:35.411940 (+ 110us) server_negotiation.cc:407] Connection header received
0806 09:34:35.412197 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:34:35.412204 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:34:35.412352 (+ 148us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:34:35.412471 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:34:35.413534 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:35.414357 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:35.416275 (+ 1918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:35.416871 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:35.417003 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:34:35.421607 (+ 4604us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:34:35.421614 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:34:35.421619 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:34:35.421671 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:34:35.425152 (+ 3481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:34:35.425944 (+ 792us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:34:35.425949 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:34:35.425953 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:34:35.426101 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:34:35.426577 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:34:35.426582 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:34:35.426585 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:34:35.426942 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:34:35.427109 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0806 09:34:35.427611 (+ 502us) server_negotiation.cc:299] Negotiation successful
0806 09:34:35.427830 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 09:34:35.434629 21442 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9017fddf7a94432b842e60d2dba5f93: Bootstrap starting.
I20260806 09:34:35.438679 21442 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c9017fddf7a94432b842e60d2dba5f93: Neither blocks nor log segments found. Creating new log.
I20260806 09:34:35.440040 21442 log.cc:824] T 00000000000000000000000000000000 P c9017fddf7a94432b842e60d2dba5f93: Log is configured to *not* fsync() on all Append() calls
I20260806 09:34:35.443394 21442 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9017fddf7a94432b842e60d2dba5f93: No bootstrap required, opened a new log
I20260806 09:34:35.450965 21442 raft_consensus.cc:348] T 00000000000000000000000000000000 P c9017fddf7a94432b842e60d2dba5f93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9017fddf7a94432b842e60d2dba5f93" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 46435 } } peers { permanent_uuid: "04ee6f7d45784380a57983f1bfb82d35" member_type: VOTER last_known_addr { host: "127.20.113.61" port: 43359 } } peers { permanent_uuid: "4f5cec1c22924a989447fcd90ff95d77" member_type: VOTER last_known_addr { host: "127.20.113.60" port: 33399 } }
I20260806 09:34:35.451339 21442 raft_consensus.cc:729] T 00000000000000000000000000000000 P c9017fddf7a94432b842e60d2dba5f93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9017fddf7a94432b842e60d2dba5f93, State: Initialized, Role: FOLLOWER
I20260806 09:34:35.451821 21442 consensus_queue.cc:260] T 00000000000000000000000000000000 P c9017fddf7a94432b842e60d2dba5f93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9017fddf7a94432b842e60d2dba5f93" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 46435 } } peers { permanent_uuid: "04ee6f7d45784380a57983f1bfb82d35" member_type: VOTER last_known_addr { host: "127.20.113.61" port: 43359 } } peers { permanent_uuid: "4f5cec1c22924a989447fcd90ff95d77" member_type: VOTER last_known_addr { host: "127.20.113.60" port: 33399 } }
I20260806 09:34:35.453769 21714 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9017fddf7a94432b842e60d2dba5f93 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9017fddf7a94432b842e60d2dba5f93" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 46435 } } peers { permanent_uuid: "04ee6f7d45784380a57983f1bfb82d35" member_type: VOTER last_known_addr { host: "127.20.113.61" port: 43359 } } peers { permanent_uuid: "4f5cec1c22924a989447fcd90ff95d77" member_type: VOTER last_known_addr { host: "127.20.113.60" port: 33399 } } }
I20260806 09:34:35.454684 21714 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9017fddf7a94432b842e60d2dba5f93 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 09:34:35.454929 21442 sys_catalog.cc:564] T 00000000000000000000000000000000 P c9017fddf7a94432b842e60d2dba5f93 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 09:34:35.455013 21405 raft_consensus.cc:482] T 00000000000000000000000000000000 P c9017fddf7a94432b842e60d2dba5f93 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260806 09:34:35.455247 21405 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c9017fddf7a94432b842e60d2dba5f93 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:34:35.459488 21405 raft_consensus.cc:504] T 00000000000000000000000000000000 P c9017fddf7a94432b842e60d2dba5f93 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9017fddf7a94432b842e60d2dba5f93" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 46435 } } peers { permanent_uuid: "04ee6f7d45784380a57983f1bfb82d35" member_type: VOTER last_known_addr { host: "127.20.113.61" port: 43359 } } peers { permanent_uuid: "4f5cec1c22924a989447fcd90ff95d77" member_type: VOTER last_known_addr { host: "127.20.113.60" port: 33399 } }
I20260806 09:34:35.463368 21513 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c9017fddf7a94432b842e60d2dba5f93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "04ee6f7d45784380a57983f1bfb82d35"
I20260806 09:34:35.464098 21513 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 04ee6f7d45784380a57983f1bfb82d35 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:34:35.467658 21627 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c9017fddf7a94432b842e60d2dba5f93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "4f5cec1c22924a989447fcd90ff95d77"
I20260806 09:34:35.468212 21627 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4f5cec1c22924a989447fcd90ff95d77 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:34:35.469285 21513 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 04ee6f7d45784380a57983f1bfb82d35 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c9017fddf7a94432b842e60d2dba5f93 in term 1.
I20260806 09:34:35.470252 21368 leader_election.cc:302] T 00000000000000000000000000000000 P c9017fddf7a94432b842e60d2dba5f93 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04ee6f7d45784380a57983f1bfb82d35, c9017fddf7a94432b842e60d2dba5f93; no voters:
I20260806 09:34:35.471951 21627 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4f5cec1c22924a989447fcd90ff95d77 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c9017fddf7a94432b842e60d2dba5f93 in term 1.
I20260806 09:34:35.472905 21405 leader_election.cc:288] T 00000000000000000000000000000000 P c9017fddf7a94432b842e60d2dba5f93 [CANDIDATE]: Term 1 election: Requested vote from peers 04ee6f7d45784380a57983f1bfb82d35 (127.20.113.61:43359), 4f5cec1c22924a989447fcd90ff95d77 (127.20.113.60:33399)
I20260806 09:34:35.473464 21714 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c9017fddf7a94432b842e60d2dba5f93 [term 1 FOLLOWER]: Leader election won for term 1
W20260806 09:34:35.478166 21370 tablet.cc:2369] T 00000000000000000000000000000000 P c9017fddf7a94432b842e60d2dba5f93: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 09:34:35.481029 21714 raft_consensus.cc:686] T 00000000000000000000000000000000 P c9017fddf7a94432b842e60d2dba5f93 [term 1 LEADER]: Becoming Leader. State: Replica: c9017fddf7a94432b842e60d2dba5f93, State: Running, Role: LEADER
I20260806 09:34:35.481998 21714 consensus_queue.cc:237] T 00000000000000000000000000000000 P c9017fddf7a94432b842e60d2dba5f93 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9017fddf7a94432b842e60d2dba5f93" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 46435 } } peers { permanent_uuid: "04ee6f7d45784380a57983f1bfb82d35" member_type: VOTER last_known_addr { host: "127.20.113.61" port: 43359 } } peers { permanent_uuid: "4f5cec1c22924a989447fcd90ff95d77" member_type: VOTER last_known_addr { host: "127.20.113.60" port: 33399 } }
I20260806 09:34:35.492599 21714 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9017fddf7a94432b842e60d2dba5f93 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c9017fddf7a94432b842e60d2dba5f93. Latest consensus state: current_term: 1 leader_uuid: "c9017fddf7a94432b842e60d2dba5f93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9017fddf7a94432b842e60d2dba5f93" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 46435 } } peers { permanent_uuid: "04ee6f7d45784380a57983f1bfb82d35" member_type: VOTER last_known_addr { host: "127.20.113.61" port: 43359 } } peers { permanent_uuid: "4f5cec1c22924a989447fcd90ff95d77" member_type: VOTER last_known_addr { host: "127.20.113.60" port: 33399 } } }
I20260806 09:34:35.493062 21714 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9017fddf7a94432b842e60d2dba5f93 [sys.catalog]: This master's current role is: LEADER
I20260806 09:34:35.517467 21734 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
W20260806 09:34:35.532147 21481 tablet.cc:2369] T 00000000000000000000000000000000 P 04ee6f7d45784380a57983f1bfb82d35: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 09:34:35.535684 21734 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 09:34:35.590847 21672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:35.490340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44778 (local address 127.20.113.62:46435)
0806 09:34:35.490522 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0806 09:34:35.490528 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:34:35.502877 (+ 12349us) server_negotiation.cc:407] Connection header received
0806 09:34:35.503166 (+ 289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:34:35.503172 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:34:35.503257 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:34:35.503334 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:34:35.505636 (+ 2302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:35.506843 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:35.540485 (+ 33642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:35.541280 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:35.541404 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:34:35.566530 (+ 25126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:34:35.566538 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:34:35.566543 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:34:35.566591 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:34:35.570420 (+ 3829us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:34:35.581486 (+ 11066us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:34:35.581493 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:34:35.581497 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:34:35.581694 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:34:35.588483 (+ 6789us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:34:35.588489 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:34:35.588493 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:34:35.589008 (+ 515us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:34:35.589387 (+ 379us) server_negotiation.cc:1035] Waiting for connection context
0806 09:34:35.590107 (+ 720us) server_negotiation.cc:299] Negotiation successful
0806 09:34:35.590430 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 09:34:35.593848 21561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:35.493670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55368 (local address 127.20.113.61:43359)
0806 09:34:35.493867 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0806 09:34:35.493872 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:34:35.520475 (+ 26603us) server_negotiation.cc:407] Connection header received
0806 09:34:35.521218 (+ 743us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:34:35.521224 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:34:35.521324 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:34:35.521402 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:34:35.523832 (+ 2430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:35.525043 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:35.544654 (+ 19611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:35.545435 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:35.545553 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:34:35.571425 (+ 25872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:34:35.571433 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:34:35.571437 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:34:35.571486 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:34:35.575559 (+ 4073us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:34:35.582291 (+ 6732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:34:35.582299 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:34:35.582303 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:34:35.582497 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:34:35.591550 (+ 9053us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:34:35.591557 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:34:35.591561 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:34:35.592074 (+ 513us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:34:35.592413 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0806 09:34:35.593108 (+ 695us) server_negotiation.cc:299] Negotiation successful
0806 09:34:35.593439 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260806 09:34:35.596697 21670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:35.491523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37096 (local address 127.20.113.60:33399)
0806 09:34:35.491765 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0806 09:34:35.491773 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 09:34:35.509917 (+ 18144us) server_negotiation.cc:407] Connection header received
0806 09:34:35.510195 (+ 278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:34:35.510201 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:34:35.510315 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:34:35.510390 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:34:35.512847 (+ 2457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:35.514001 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:35.542700 (+ 28699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:35.543485 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:35.543599 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:34:35.576690 (+ 33091us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:34:35.576698 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:34:35.576703 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:34:35.576748 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:34:35.580521 (+ 3773us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:34:35.583037 (+ 2516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:34:35.583043 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:34:35.583047 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:34:35.583225 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:34:35.594502 (+ 11277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:34:35.594509 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:34:35.594514 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:34:35.595013 (+ 499us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:34:35.595309 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0806 09:34:35.595940 (+ 631us) server_negotiation.cc:299] Negotiation successful
0806 09:34:35.596236 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260806 09:34:35.619413 21513 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 04ee6f7d45784380a57983f1bfb82d35 [term 1 FOLLOWER]: Refusing update from remote peer c9017fddf7a94432b842e60d2dba5f93: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 09:34:35.622313 21627 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4f5cec1c22924a989447fcd90ff95d77 [term 1 FOLLOWER]: Refusing update from remote peer c9017fddf7a94432b842e60d2dba5f93: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 09:34:35.622426 21714 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c9017fddf7a94432b842e60d2dba5f93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "04ee6f7d45784380a57983f1bfb82d35" member_type: VOTER last_known_addr { host: "127.20.113.61" port: 43359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 09:34:35.623466 21714 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c9017fddf7a94432b842e60d2dba5f93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f5cec1c22924a989447fcd90ff95d77" member_type: VOTER last_known_addr { host: "127.20.113.60" port: 33399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 09:34:35.652048 21675 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f5cec1c22924a989447fcd90ff95d77 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c9017fddf7a94432b842e60d2dba5f93. Latest consensus state: current_term: 1 leader_uuid: "c9017fddf7a94432b842e60d2dba5f93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9017fddf7a94432b842e60d2dba5f93" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 46435 } } peers { permanent_uuid: "04ee6f7d45784380a57983f1bfb82d35" member_type: VOTER last_known_addr { host: "127.20.113.61" port: 43359 } } peers { permanent_uuid: "4f5cec1c22924a989447fcd90ff95d77" member_type: VOTER last_known_addr { host: "127.20.113.60" port: 33399 } } }
I20260806 09:34:35.652405 21675 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f5cec1c22924a989447fcd90ff95d77 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 09:34:35.653913 21749 mvcc.cc:205] Tried to move back new op lower bound from 7315492354501890048 to 7315492353984016384. Current Snapshot: MvccSnapshot[applied={T|T < 7315492354501890048}]
I20260806 09:34:35.655377 21694 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04ee6f7d45784380a57983f1bfb82d35 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c9017fddf7a94432b842e60d2dba5f93. Latest consensus state: current_term: 1 leader_uuid: "c9017fddf7a94432b842e60d2dba5f93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9017fddf7a94432b842e60d2dba5f93" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 46435 } } peers { permanent_uuid: "04ee6f7d45784380a57983f1bfb82d35" member_type: VOTER last_known_addr { host: "127.20.113.61" port: 43359 } } peers { permanent_uuid: "4f5cec1c22924a989447fcd90ff95d77" member_type: VOTER last_known_addr { host: "127.20.113.60" port: 33399 } } }
I20260806 09:34:35.655746 21694 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04ee6f7d45784380a57983f1bfb82d35 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 09:34:35.656216 21753 mvcc.cc:205] Tried to move back new op lower bound from 7315492354501890048 to 7315492353984016384. Current Snapshot: MvccSnapshot[applied={T|T < 7315492354501890048}]
I20260806 09:34:35.660162 21754 mvcc.cc:205] Tried to move back new op lower bound from 7315492354501890048 to 7315492353984016384. Current Snapshot: MvccSnapshot[applied={T|T < 7315492354501890048}]
I20260806 09:34:35.667115 21719 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9017fddf7a94432b842e60d2dba5f93 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c9017fddf7a94432b842e60d2dba5f93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9017fddf7a94432b842e60d2dba5f93" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 46435 } } peers { permanent_uuid: "04ee6f7d45784380a57983f1bfb82d35" member_type: VOTER last_known_addr { host: "127.20.113.61" port: 43359 } } peers { permanent_uuid: "4f5cec1c22924a989447fcd90ff95d77" member_type: VOTER last_known_addr { host: "127.20.113.60" port: 33399 } } }
I20260806 09:34:35.667567 21719 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9017fddf7a94432b842e60d2dba5f93 [sys.catalog]: This master's current role is: LEADER
I20260806 09:34:35.667943 21719 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9017fddf7a94432b842e60d2dba5f93 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c9017fddf7a94432b842e60d2dba5f93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9017fddf7a94432b842e60d2dba5f93" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 46435 } } peers { permanent_uuid: "04ee6f7d45784380a57983f1bfb82d35" member_type: VOTER last_known_addr { host: "127.20.113.61" port: 43359 } } peers { permanent_uuid: "4f5cec1c22924a989447fcd90ff95d77" member_type: VOTER last_known_addr { host: "127.20.113.60" port: 33399 } } }
I20260806 09:34:35.668181 21719 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9017fddf7a94432b842e60d2dba5f93 [sys.catalog]: This master's current role is: LEADER
I20260806 09:34:35.668949 21694 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04ee6f7d45784380a57983f1bfb82d35 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c9017fddf7a94432b842e60d2dba5f93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9017fddf7a94432b842e60d2dba5f93" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 46435 } } peers { permanent_uuid: "04ee6f7d45784380a57983f1bfb82d35" member_type: VOTER last_known_addr { host: "127.20.113.61" port: 43359 } } peers { permanent_uuid: "4f5cec1c22924a989447fcd90ff95d77" member_type: VOTER last_known_addr { host: "127.20.113.60" port: 33399 } } }
I20260806 09:34:35.669198 21694 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04ee6f7d45784380a57983f1bfb82d35 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 09:34:35.675323 21734 catalog_manager.cc:1347] Generated new cluster ID: b78c6005d1e94d969decf15f6469c198
I20260806 09:34:35.675436 21734 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 09:34:35.676729 21675 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f5cec1c22924a989447fcd90ff95d77 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c9017fddf7a94432b842e60d2dba5f93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9017fddf7a94432b842e60d2dba5f93" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 46435 } } peers { permanent_uuid: "04ee6f7d45784380a57983f1bfb82d35" member_type: VOTER last_known_addr { host: "127.20.113.61" port: 43359 } } peers { permanent_uuid: "4f5cec1c22924a989447fcd90ff95d77" member_type: VOTER last_known_addr { host: "127.20.113.60" port: 33399 } } }
I20260806 09:34:35.677007 21675 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f5cec1c22924a989447fcd90ff95d77 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 09:34:35.713155 21734 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 09:34:35.715166 21734 catalog_manager.cc:1504] Loading token signing keys...
I20260806 09:34:35.738147 21734 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c9017fddf7a94432b842e60d2dba5f93: Generated new TSK 0
I20260806 09:34:35.739470 21734 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 09:34:35.898815 21672 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 09:34:35.883710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44790 (local address 127.20.113.62:46435)
0806 09:34:35.883936 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0806 09:34:35.883943 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:34:35.895859 (+ 11916us) server_negotiation.cc:407] Connection header received
0806 09:34:35.896090 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:34:35.896098 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:34:35.896198 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:34:35.896330 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:34:35.898408 (+ 2078us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44790: BlockingRecv error: recv got EOF from 127.0.0.1:44790 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
W20260806 09:34:35.902259 21670 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 09:34:35.890411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37108 (local address 127.20.113.60:33399)
0806 09:34:35.890585 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0806 09:34:35.890591 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:34:35.896515 (+ 5924us) server_negotiation.cc:407] Connection header received
0806 09:34:35.896695 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:34:35.896704 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:34:35.896811 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:34:35.896958 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:34:35.901893 (+ 4935us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37108: BlockingRecv error: recv got EOF from 127.0.0.1:37108 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260806 09:34:35.904882 21561 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 09:34:35.895093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55374 (local address 127.20.113.61:43359)
0806 09:34:35.895316 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0806 09:34:35.895323 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:34:35.900379 (+ 5056us) server_negotiation.cc:407] Connection header received
0806 09:34:35.903133 (+ 2754us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:34:35.903142 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:34:35.903308 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:34:35.903404 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:34:35.904626 (+ 1222us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55374: BlockingRecv error: recv got EOF from 127.0.0.1:55374 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
W20260806 09:34:35.911979 21561 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 09:34:35.910084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55378 (local address 127.20.113.61:43359)
0806 09:34:35.910297 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0806 09:34:35.910303 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:34:35.910572 (+ 269us) server_negotiation.cc:407] Connection header received
0806 09:34:35.910809 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:34:35.910821 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:34:35.910926 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:34:35.911033 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:34:35.911728 (+ 695us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55378: BlockingRecv error: recv got EOF from 127.0.0.1:55378 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260806 09:34:35.918795 21561 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 09:34:35.917011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55392 (local address 127.20.113.61:43359)
0806 09:34:35.917202 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0806 09:34:35.917208 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:34:35.917428 (+ 220us) server_negotiation.cc:407] Connection header received
0806 09:34:35.917600 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:34:35.917609 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:34:35.917681 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:34:35.917791 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:34:35.918572 (+ 781us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55392: BlockingRecv error: recv got EOF from 127.0.0.1:55392 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
W20260806 09:34:35.926486 21561 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 09:34:35.924414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55404 (local address 127.20.113.61:43359)
0806 09:34:35.924572 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0806 09:34:35.924576 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 09:34:35.925190 (+ 614us) server_negotiation.cc:407] Connection header received
0806 09:34:35.925403 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:34:35.925408 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:34:35.925504 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:34:35.925622 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:34:35.926280 (+ 658us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55404: BlockingRecv error: recv got EOF from 127.0.0.1:55404 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
W20260806 09:34:35.935763 21561 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 09:34:35.933679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55420 (local address 127.20.113.61:43359)
0806 09:34:35.933991 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0806 09:34:35.933999 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 09:34:35.934477 (+ 478us) server_negotiation.cc:407] Connection header received
0806 09:34:35.934676 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:34:35.934682 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:34:35.934755 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:34:35.934854 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:34:35.935534 (+ 680us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55420: BlockingRecv error: recv got EOF from 127.0.0.1:55420 (error 108)
Metrics: {"server-negotiator.queue_time_us":127}
W20260806 09:34:35.949200 21561 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 09:34:35.946706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55436 (local address 127.20.113.61:43359)
0806 09:34:35.946977 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0806 09:34:35.946987 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 09:34:35.947547 (+ 560us) server_negotiation.cc:407] Connection header received
0806 09:34:35.947875 (+ 328us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:34:35.947881 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:34:35.947965 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:34:35.948075 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:34:35.948938 (+ 863us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55436: BlockingRecv error: recv got EOF from 127.0.0.1:55436 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
W20260806 09:34:35.970824 21561 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 09:34:35.968456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55452 (local address 127.20.113.61:43359)
0806 09:34:35.968697 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0806 09:34:35.968706 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 09:34:35.969298 (+ 592us) server_negotiation.cc:407] Connection header received
0806 09:34:35.969532 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:34:35.969537 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:34:35.969611 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:34:35.969720 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:34:35.970535 (+ 815us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55452: BlockingRecv error: recv got EOF from 127.0.0.1:55452 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
W20260806 09:34:36.016245 21561 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 09:34:36.014024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55468 (local address 127.20.113.61:43359)
0806 09:34:36.014220 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0806 09:34:36.014229 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 09:34:36.014614 (+ 385us) server_negotiation.cc:407] Connection header received
0806 09:34:36.014819 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:34:36.014825 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:34:36.014932 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:34:36.015054 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:34:36.015961 (+ 907us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55468: BlockingRecv error: recv got EOF from 127.0.0.1:55468 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260806 09:34:36.085902 21561 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 09:34:36.083798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55474 (local address 127.20.113.61:43359)
0806 09:34:36.084040 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0806 09:34:36.084048 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 09:34:36.084481 (+ 433us) server_negotiation.cc:407] Connection header received
0806 09:34:36.084669 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:34:36.084675 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:34:36.084778 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:34:36.084906 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:34:36.085660 (+ 754us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55474: BlockingRecv error: recv got EOF from 127.0.0.1:55474 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
W20260806 09:34:36.220073 21561 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 09:34:36.218029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55490 (local address 127.20.113.61:43359)
0806 09:34:36.218243 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0806 09:34:36.218248 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:34:36.218722 (+ 474us) server_negotiation.cc:407] Connection header received
0806 09:34:36.218931 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:34:36.218935 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:34:36.219007 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:34:36.219103 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:34:36.219877 (+ 774us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55490: BlockingRecv error: recv got EOF from 127.0.0.1:55490 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 09:34:36.264245 21689 catalog_manager.cc:1259] Loaded cluster ID: b78c6005d1e94d969decf15f6469c198
I20260806 09:34:36.264427 21689 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 4f5cec1c22924a989447fcd90ff95d77: loading cluster ID for follower catalog manager: success
I20260806 09:34:36.268260 21689 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 4f5cec1c22924a989447fcd90ff95d77: acquiring CA information for follower catalog manager: success
I20260806 09:34:36.270054 21689 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 4f5cec1c22924a989447fcd90ff95d77: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 09:34:36.341351 21710 catalog_manager.cc:1259] Loaded cluster ID: b78c6005d1e94d969decf15f6469c198
I20260806 09:34:36.341459 21710 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 04ee6f7d45784380a57983f1bfb82d35: loading cluster ID for follower catalog manager: success
I20260806 09:34:36.343963 21710 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 04ee6f7d45784380a57983f1bfb82d35: acquiring CA information for follower catalog manager: success
I20260806 09:34:36.345777 21710 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 04ee6f7d45784380a57983f1bfb82d35: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 09:34:36.490178 21561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:36.484246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55506 (local address 127.20.113.61:43359)
0806 09:34:36.484504 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0806 09:34:36.484535 (+ 31us) server_negotiation.cc:399] Waiting for connection header
0806 09:34:36.484874 (+ 339us) server_negotiation.cc:407] Connection header received
0806 09:34:36.485068 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:34:36.485072 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:34:36.485253 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:34:36.485369 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 09:34:36.485851 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:36.486636 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:36.487905 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:36.488364 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:36.488463 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:34:36.488921 (+ 458us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 09:34:36.489474 (+ 553us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 09:34:36.489611 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0806 09:34:36.489762 (+ 151us) server_negotiation.cc:299] Negotiation successful
0806 09:34:36.489890 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260806 09:34:36.501283 21851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:36.496076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37118 (local address 127.20.113.60:33399)
0806 09:34:36.496552 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0806 09:34:36.496556 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 09:34:36.496613 (+ 57us) server_negotiation.cc:407] Connection header received
0806 09:34:36.496798 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:34:36.496802 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:34:36.496946 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:34:36.497084 (+ 138us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 09:34:36.497402 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:36.498090 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:36.499348 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:36.499727 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:36.499821 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:34:36.500220 (+ 399us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 09:34:36.500743 (+ 523us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 09:34:36.500845 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 09:34:36.500957 (+ 112us) server_negotiation.cc:299] Negotiation successful
0806 09:34:36.501028 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":381,"thread_start_us":279,"threads_started":1}
I20260806 09:34:36.526916 21561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:36.507873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55514 (local address 127.20.113.61:43359)
0806 09:34:36.508087 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0806 09:34:36.508096 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 09:34:36.521118 (+ 13022us) server_negotiation.cc:407] Connection header received
0806 09:34:36.521403 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:34:36.521409 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:34:36.521502 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:34:36.521590 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 09:34:36.522145 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:36.522865 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:36.524913 (+ 2048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:36.525347 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:36.525486 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:34:36.526120 (+ 634us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 09:34:36.526254 (+ 134us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 09:34:36.526309 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 09:34:36.526546 (+ 237us) server_negotiation.cc:299] Negotiation successful
0806 09:34:36.526625 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260806 09:34:36.527319 21851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:36.508067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37134 (local address 127.20.113.60:33399)
0806 09:34:36.513579 (+ 5512us) server_negotiation.cc:206] Beginning negotiation
0806 09:34:36.513587 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 09:34:36.520412 (+ 6825us) server_negotiation.cc:407] Connection header received
0806 09:34:36.520600 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:34:36.520608 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:34:36.520713 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:34:36.520823 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 09:34:36.521883 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:36.522929 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:36.525499 (+ 2570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:36.525921 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:36.526057 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:34:36.526529 (+ 472us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 09:34:36.526660 (+ 131us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 09:34:36.526723 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 09:34:36.526936 (+ 213us) server_negotiation.cc:299] Negotiation successful
0806 09:34:36.527012 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5364}
I20260806 09:34:36.528872 21859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:36.507581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44792 (local address 127.20.113.62:46435)
0806 09:34:36.519121 (+ 11540us) server_negotiation.cc:206] Beginning negotiation
0806 09:34:36.519131 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 09:34:36.520365 (+ 1234us) server_negotiation.cc:407] Connection header received
0806 09:34:36.520537 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:34:36.520545 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:34:36.520799 (+ 254us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:34:36.520994 (+ 195us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 09:34:36.521629 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:36.522830 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:36.525478 (+ 2648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:36.526166 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:36.526290 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:34:36.527516 (+ 1226us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 09:34:36.527948 (+ 432us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 09:34:36.528069 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0806 09:34:36.528552 (+ 483us) server_negotiation.cc:299] Negotiation successful
0806 09:34:36.528642 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11372,"thread_start_us":11558,"threads_started":1}
I20260806 09:34:36.533336 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 21342
I20260806 09:34:36.534502 21436 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:34:36.642789 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 21342
I20260806 09:34:36.670683 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 21453
I20260806 09:34:36.671833 21546 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:34:36.781881 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 21453
I20260806 09:34:36.809978 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 21564
I20260806 09:34:36.811153 21656 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:34:36.920616 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 21564
2026-08-06T09:34:36Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (5158 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786008495763054-20932-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 09:34:37 dist-test-slave-45dx krb5kdc[21873](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 09:34:37 dist-test-slave-45dx krb5kdc[21873](info): set up 2 sockets
Aug 06 09:34:37 dist-test-slave-45dx krb5kdc[21873](info): commencing operation
krb5kdc: starting...
W20260806 09:34:38.259874 20932 mini_kdc.cc:121] Time spent starting KDC: real 1.287s user 0.000s sys 0.059s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786008495763054-20932-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786008495763054-20932-0/krb5kdc/test-user.keytab.
Aug 06 09:34:38 dist-test-slave-45dx krb5kdc[21873](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008878, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T09:34:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T09:34:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.113.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.113.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.113.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab.
Entry for principal kudu/127.20.113.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab.
I20260806 09:34:38.631807 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786008495763054-20932-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786008495763054-20932-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786008495763054-20932-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.113.62:32933
--webserver_interface=127.20.113.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab
--principal=kudu/127.20.113.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:33177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.113.62:32933,127.20.113.61:45653,127.20.113.60:37157
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 09:34:38.977878 21894 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 09:34:38.978188 21894 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:34:38.978335 21894 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:34:38.978525 21894 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:34:38.985540 21894 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 09:34:38.985621 21894 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:34:38.985673 21894 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:34:38.985723 21894 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 09:34:38.985780 21894 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 09:34:38.995029 21894 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:33177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786008495763054-20932-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786008495763054-20932-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.113.62:32933,127.20.113.61:45653,127.20.113.60:37157
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.62.keytab
--principal=kudu/127.20.113.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.113.62:32933
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.113.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786008495763054-20932-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:34:38.996186 21894 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:34:38.997500 21894 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:34:39.009132 21904 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:34:39.009084 21909 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:34:39.009193 21903 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:34:39.015137 21894 server_base.cc:1034] running on GCE node
I20260806 09:34:39.015913 21894 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:34:39.017324 21894 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:34:39.018579 21894 hybrid_clock.cc:648] HybridClock initialized: now 1786008879018525 us; error 28 us; skew 500 ppm
I20260806 09:34:39.018913 21894 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:34:39 dist-test-slave-45dx krb5kdc[21873](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008879, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:34:39.022130 21894 init.cc:377] Logged in from keytab as kudu/127.20.113.62@KRBTEST.COM (short username kudu)
I20260806 09:34:39.023535 21894 webserver.cc:460] Webserver started at http://127.20.113.62:42983/ using document root <none> and password file <none>
I20260806 09:34:39.024278 21894 fs_manager.cc:359] Metadata directory not provided
I20260806 09:34:39.024470 21894 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:34:39.024811 21894 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:34:39.027801 21894 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786008495763054-20932-0/minicluster-data/master-0/data/instance:
uuid: "d5cfae47be6541ef8b7717525d447fd7"
format_stamp: "Formatted at 2026-08-06 09:34:39 on dist-test-slave-45dx"
server_key: "b0df7146cb685bb50fb9a0815b53a096"
server_key_iv: "e670c86e0cd64aab4183efc74fce7920"
server_key_version: "encryptionkey@0"
I20260806 09:34:39.028697 21894 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786008495763054-20932-0/minicluster-data/master-0/wal/instance:
uuid: "d5cfae47be6541ef8b7717525d447fd7"
format_stamp: "Formatted at 2026-08-06 09:34:39 on dist-test-slave-45dx"
server_key: "b0df7146cb685bb50fb9a0815b53a096"
server_key_iv: "e670c86e0cd64aab4183efc74fce7920"
server_key_version: "encryptionkey@0"
I20260806 09:34:39.033591 21894 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260806 09:34:39.036980 21918 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:34:39.038269 21894 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 09:34:39.038465 21894 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786008495763054-20932-0/minicluster-data/master-0/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786008495763054-20932-0/minicluster-data/master-0/wal
uuid: "d5cfae47be6541ef8b7717525d447fd7"
format_stamp: "Formatted at 2026-08-06 09:34:39 on dist-test-slave-45dx"
server_key: "b0df7146cb685bb50fb9a0815b53a096"
server_key_iv: "e670c86e0cd64aab4183efc74fce7920"
server_key_version: "encryptionkey@0"
I20260806 09:34:39.038684 21894 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786008495763054-20932-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786008495763054-20932-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786008495763054-20932-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:34:39.064961 21894 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:34:39.069237 21894 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:34:39.069520 21894 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:34:39.088637 21894 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.62:32933
I20260806 09:34:39.088685 21992 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.62:32933 every 8 connection(s)
I20260806 09:34:39.089943 21894 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786008495763054-20932-0/minicluster-data/master-0/data/info.pb
I20260806 09:34:39.093568 21994 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:34:39.098505 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 21894
I20260806 09:34:39.098707 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786008495763054-20932-0/minicluster-data/master-0/wal/instance
I20260806 09:34:39.099234 20932 external_mini_cluster.cc:1428] Setting key 9af55b6ce142719f25938aab71798abc
I20260806 09:34:39.100657 21994 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.113.62" port: 32933 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 09:34:39 dist-test-slave-45dx krb5kdc[21873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008879, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.62@KRBTEST.COM for kudu/127.20.113.62@KRBTEST.COM
I20260806 09:34:39.126626 21997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:39.104468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:32933 (local address 127.0.0.1:51138)
0806 09:34:39.105382 (+ 914us) negotiation.cc:107] Waiting for socket to connect
0806 09:34:39.105442 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0806 09:34:39.108269 (+ 2827us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:34:39.109130 (+ 861us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:34:39.109139 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:34:39.110013 (+ 874us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:34:39.110512 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:34:39.110539 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:34:39.112182 (+ 1643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:34:39.112190 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:34:39.114046 (+ 1856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:34:39.114063 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:34:39.115049 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:34:39.115058 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:34:39.115277 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:34:39.115989 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:34:39.116027 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:34:39.118581 (+ 2554us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:34:39.122803 (+ 4222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:34:39.122813 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:34:39.122822 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:34:39.123202 (+ 380us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:34:39.123689 (+ 487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:34:39.123692 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:34:39.123694 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:34:39.123836 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:34:39.124851 (+ 1015us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:34:39.124857 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:34:39.125055 (+ 198us) client_negotiation.cc:769] Sending connection context
0806 09:34:39.125245 (+ 190us) client_negotiation.cc:240] Negotiation successful
0806 09:34:39.125602 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":608,"thread_start_us":376,"threads_started":1}
I20260806 09:34:39.126664 21998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:39.104468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51138 (local address 127.20.113.62:32933)
0806 09:34:39.106373 (+ 1905us) server_negotiation.cc:206] Beginning negotiation
0806 09:34:39.106379 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:34:39.106427 (+ 48us) server_negotiation.cc:407] Connection header received
0806 09:34:39.108684 (+ 2257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:34:39.108689 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:34:39.108905 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:34:39.109053 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:34:39.110863 (+ 1810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:39.112021 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:39.114263 (+ 2242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:39.114797 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:39.114930 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:34:39.118825 (+ 3895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:34:39.118832 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:34:39.118843 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:34:39.118873 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:34:39.122661 (+ 3788us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:34:39.123351 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:34:39.123362 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:34:39.123368 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:34:39.123580 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:34:39.123961 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:34:39.123969 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:34:39.123975 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:34:39.124721 (+ 746us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:34:39.124932 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0806 09:34:39.125441 (+ 509us) server_negotiation.cc:299] Negotiation successful
0806 09:34:39.125822 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":56,"server-negotiator.queue_time_us":1666,"spinlock_wait_cycles":15232,"thread_start_us":1439,"threads_started":1}
I20260806 09:34:39.129729 21994 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.113.61" port: 45653 } has no permanent_uuid. Determining permanent_uuid...
I20260806 09:34:39.130725 21997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:39.130333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.61:45653 (local address 127.0.0.1:41114)
0806 09:34:39.130511 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0806 09:34:39.130629 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.61:45653: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260806 09:34:39.130995 21923 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.113.61:45653: connect: Connection refused (error 111)
I20260806 09:34:39.132537 21997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:39.132120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.61:45653 (local address 127.0.0.1:41130)
0806 09:34:39.132285 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0806 09:34:39.132423 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.61:45653: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260806 09:34:39.132737 21994 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.113.61:45653: Network error: Client connection negotiation failed: client connection to 127.20.113.61:45653: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.20.113.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.113.61@KRBTEST.COM" created.
I20260806 09:34:39.182058 21994 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.113.61" port: 45653 } attempt: 1
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.113.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.61.keytab.
Entry for principal kudu/127.20.113.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.61.keytab.
I20260806 09:34:39.183430 21997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:39.182860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.61:45653 (local address 127.0.0.1:41140)
0806 09:34:39.183088 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0806 09:34:39.183221 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.61:45653: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260806 09:34:39.183849 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786008495763054-20932-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786008495763054-20932-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786008495763054-20932-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786008495763054-20932-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.113.61:45653
--webserver_interface=127.20.113.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.61.keytab
--principal=kudu/127.20.113.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:33177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.113.62:32933,127.20.113.61:45653,127.20.113.60:37157
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 09:34:39.184643 21997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:39.184168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.61:45653 (local address 127.0.0.1:41154)
0806 09:34:39.184381 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0806 09:34:39.184473 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.61:45653: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260806 09:34:39.184864 21994 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.113.61:45653: Network error: Client connection negotiation failed: client connection to 127.20.113.61:45653: connect: Connection refused (error 111)
I20260806 09:34:39.253105 21994 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.113.61" port: 45653 } attempt: 2
I20260806 09:34:39.254290 21997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:39.253789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.61:45653 (local address 127.0.0.1:41168)
0806 09:34:39.254051 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0806 09:34:39.254139 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.61:45653: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260806 09:34:39.255303 21997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:39.254959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.61:45653 (local address 127.0.0.1:41180)
0806 09:34:39.255104 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0806 09:34:39.255188 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.61:45653: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260806 09:34:39.255568 21994 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.113.61:45653: Network error: Client connection negotiation failed: client connection to 127.20.113.61:45653: connect: Connection refused (error 111)
I20260806 09:34:39.346958 21994 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.113.61" port: 45653 } attempt: 3
I20260806 09:34:39.348130 21997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:39.347673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.61:45653 (local address 127.0.0.1:41194)
0806 09:34:39.347893 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0806 09:34:39.347983 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.61:45653: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260806 09:34:39.349174 21997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:39.348830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.61:45653 (local address 127.0.0.1:41210)
0806 09:34:39.348981 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0806 09:34:39.349049 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.61:45653: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260806 09:34:39.349423 21994 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.113.61:45653: Network error: Client connection negotiation failed: client connection to 127.20.113.61:45653: connect: Connection refused (error 111)
I20260806 09:34:39.492712 21994 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.113.61" port: 45653 } attempt: 4
I20260806 09:34:39.494086 21997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:39.493511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.61:45653 (local address 127.0.0.1:41216)
0806 09:34:39.493848 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0806 09:34:39.493950 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.61:45653: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187}
I20260806 09:34:39.495078 21997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:39.494751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.61:45653 (local address 127.0.0.1:41228)
0806 09:34:39.494892 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0806 09:34:39.494973 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.61:45653: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260806 09:34:39.495316 21994 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.113.61:45653: Network error: Client connection negotiation failed: client connection to 127.20.113.61:45653: connect: Connection refused (error 111)
W20260806 09:34:39.528570 22003 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 09:34:39.528860 22003 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:34:39.528934 22003 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:34:39.529012 22003 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:34:39.535931 22003 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 09:34:39.536016 22003 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:34:39.536072 22003 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:34:39.536113 22003 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 09:34:39.536167 22003 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 09:34:39.545819 22003 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:33177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786008495763054-20932-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786008495763054-20932-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.20.113.62:32933,127.20.113.61:45653,127.20.113.60:37157
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.61.keytab
--principal=kudu/127.20.113.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.113.61:45653
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786008495763054-20932-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.20.113.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786008495763054-20932-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:34:39.546993 22003 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:34:39.548259 22003 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:34:39.560410 22013 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:34:39.560541 22012 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:34:39.560809 22018 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:34:39.561697 22003 server_base.cc:1034] running on GCE node
I20260806 09:34:39.562497 22003 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:34:39.563828 22003 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:34:39.565086 22003 hybrid_clock.cc:648] HybridClock initialized: now 1786008879565017 us; error 40 us; skew 500 ppm
I20260806 09:34:39.565418 22003 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:34:39 dist-test-slave-45dx krb5kdc[21873](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008879, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:34:39.568620 22003 init.cc:377] Logged in from keytab as kudu/127.20.113.61@KRBTEST.COM (short username kudu)
I20260806 09:34:39.569953 22003 webserver.cc:460] Webserver started at http://127.20.113.61:38341/ using document root <none> and password file <none>
I20260806 09:34:39.570607 22003 fs_manager.cc:359] Metadata directory not provided
I20260806 09:34:39.570696 22003 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:34:39.570966 22003 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:34:39.573907 22003 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786008495763054-20932-0/minicluster-data/master-1/data/instance:
uuid: "18af5fcd3b6747348add713a9330a7b4"
format_stamp: "Formatted at 2026-08-06 09:34:39 on dist-test-slave-45dx"
server_key: "c6e736e87fd50903f44899c78355a59e"
server_key_iv: "af668b6d9506212939da370135782708"
server_key_version: "encryptionkey@0"
I20260806 09:34:39.574682 22003 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786008495763054-20932-0/minicluster-data/master-1/wal/instance:
uuid: "18af5fcd3b6747348add713a9330a7b4"
format_stamp: "Formatted at 2026-08-06 09:34:39 on dist-test-slave-45dx"
server_key: "c6e736e87fd50903f44899c78355a59e"
server_key_iv: "af668b6d9506212939da370135782708"
server_key_version: "encryptionkey@0"
I20260806 09:34:39.579698 22003 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 09:34:39.583200 22033 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:34:39.584532 22003 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 09:34:39.584663 22003 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786008495763054-20932-0/minicluster-data/master-1/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786008495763054-20932-0/minicluster-data/master-1/wal
uuid: "18af5fcd3b6747348add713a9330a7b4"
format_stamp: "Formatted at 2026-08-06 09:34:39 on dist-test-slave-45dx"
server_key: "c6e736e87fd50903f44899c78355a59e"
server_key_iv: "af668b6d9506212939da370135782708"
server_key_version: "encryptionkey@0"
I20260806 09:34:39.584833 22003 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786008495763054-20932-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786008495763054-20932-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786008495763054-20932-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:34:39.597591 22003 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:34:39.601346 22003 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:34:39.601563 22003 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:34:39.620244 22003 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.61:45653
I20260806 09:34:39.620262 22102 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.61:45653 every 8 connection(s)
I20260806 09:34:39.621575 22003 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786008495763054-20932-0/minicluster-data/master-1/data/info.pb
I20260806 09:34:39.625204 22103 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:34:39.629222 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 22003
I20260806 09:34:39.629431 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786008495763054-20932-0/minicluster-data/master-1/wal/instance
I20260806 09:34:39.629988 20932 external_mini_cluster.cc:1428] Setting key eccd1cc255ff2329de62b3eda97f8fb4
I20260806 09:34:39.633750 22103 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.113.62" port: 32933 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 09:34:39 dist-test-slave-45dx krb5kdc[21873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008879, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.61@KRBTEST.COM for kudu/127.20.113.62@KRBTEST.COM
I20260806 09:34:39.661415 22105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:39.637110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51148 (local address 127.20.113.62:32933)
0806 09:34:39.642933 (+ 5823us) server_negotiation.cc:206] Beginning negotiation
0806 09:34:39.642942 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 09:34:39.642971 (+ 29us) server_negotiation.cc:407] Connection header received
0806 09:34:39.644623 (+ 1652us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:34:39.644629 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:34:39.644728 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:34:39.644872 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:34:39.646770 (+ 1898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:39.647532 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:39.649124 (+ 1592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:39.649659 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:39.649811 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:34:39.653898 (+ 4087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:34:39.653907 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:34:39.653915 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:34:39.653967 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:34:39.657849 (+ 3882us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:34:39.658613 (+ 764us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:34:39.658621 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:34:39.658627 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:34:39.658779 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:34:39.659342 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:34:39.659349 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:34:39.659355 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:34:39.659738 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:34:39.659899 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0806 09:34:39.660618 (+ 719us) server_negotiation.cc:299] Negotiation successful
0806 09:34:39.660901 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5690,"thread_start_us":5545,"threads_started":1}
I20260806 09:34:39.661921 22108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:39.637502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:32933 (local address 127.0.0.1:51148)
0806 09:34:39.642287 (+ 4785us) negotiation.cc:107] Waiting for socket to connect
0806 09:34:39.642344 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0806 09:34:39.644336 (+ 1992us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:34:39.645272 (+ 936us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:34:39.645285 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:34:39.646033 (+ 748us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:34:39.646553 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:34:39.646569 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:34:39.647694 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:34:39.647699 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:34:39.648965 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:34:39.648978 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:34:39.649935 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:34:39.649945 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:34:39.650273 (+ 328us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:34:39.651158 (+ 885us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:34:39.651201 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:34:39.653662 (+ 2461us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:34:39.657996 (+ 4334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:34:39.658002 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:34:39.658013 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:34:39.658442 (+ 429us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:34:39.659010 (+ 568us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:34:39.659014 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:34:39.659017 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:34:39.659189 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:34:39.659860 (+ 671us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:34:39.659878 (+ 18us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:34:39.660181 (+ 303us) client_negotiation.cc:769] Sending connection context
0806 09:34:39.660478 (+ 297us) client_negotiation.cc:240] Negotiation successful
0806 09:34:39.660830 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4442,"thread_start_us":4234,"threads_started":1}
I20260806 09:34:39.663687 22103 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.113.61" port: 45653 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.20.113.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.113.60@KRBTEST.COM" created.
Aug 06 09:34:39 dist-test-slave-45dx krb5kdc[21873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008879, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.61@KRBTEST.COM for kudu/127.20.113.61@KRBTEST.COM
I20260806 09:34:39.682590 22111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:39.664774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41234 (local address 127.20.113.61:45653)
0806 09:34:39.665523 (+ 749us) server_negotiation.cc:206] Beginning negotiation
0806 09:34:39.665529 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:34:39.665565 (+ 36us) server_negotiation.cc:407] Connection header received
0806 09:34:39.665604 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:34:39.665609 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:34:39.665757 (+ 148us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:34:39.665890 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:34:39.666915 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:39.667959 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:39.669782 (+ 1823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:39.670277 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:39.670370 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:34:39.676890 (+ 6520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:34:39.676902 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:34:39.676913 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:34:39.676947 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:34:39.679642 (+ 2695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:34:39.680452 (+ 810us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:34:39.680458 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:34:39.680460 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:34:39.680574 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:34:39.681007 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:34:39.681010 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:34:39.681012 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:34:39.681323 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:34:39.681482 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0806 09:34:39.681924 (+ 442us) server_negotiation.cc:299] Negotiation successful
0806 09:34:39.682281 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":153,"server-negotiator.queue_time_us":605,"thread_start_us":457,"threads_started":1}
I20260806 09:34:39.682629 22108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:39.664494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.61:45653 (local address 127.0.0.1:41234)
0806 09:34:39.664707 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0806 09:34:39.664727 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 09:34:39.664829 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:34:39.665944 (+ 1115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:34:39.665951 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:34:39.666497 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:34:39.666732 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:34:39.666748 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:34:39.668103 (+ 1355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:34:39.668113 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:34:39.669638 (+ 1525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:34:39.669657 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:34:39.670453 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:34:39.670459 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:34:39.670649 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:34:39.671276 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:34:39.671301 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:34:39.676673 (+ 5372us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:34:39.679785 (+ 3112us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:34:39.679792 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:34:39.679798 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:34:39.680330 (+ 532us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:34:39.680684 (+ 354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:34:39.680691 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:34:39.680696 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:34:39.680897 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:34:39.681454 (+ 557us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:34:39.681460 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:34:39.681620 (+ 160us) client_negotiation.cc:769] Sending connection context
0806 09:34:39.681791 (+ 171us) client_negotiation.cc:240] Negotiation successful
0806 09:34:39.682158 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68}
I20260806 09:34:39.685324 22103 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.113.60" port: 37157 } has no permanent_uuid. Determining permanent_uuid...
I20260806 09:34:39.686488 22108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:39.685928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.60:37157 (local address 127.0.0.1:36704)
0806 09:34:39.686126 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0806 09:34:39.686280 (+ 154us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.60:37157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260806 09:34:39.686769 22035 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.113.60:37157: connect: Connection refused (error 111)
I20260806 09:34:39.688591 22108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:39.688076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.60:37157 (local address 127.0.0.1:36714)
0806 09:34:39.688244 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0806 09:34:39.688446 (+ 202us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.60:37157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260806 09:34:39.688822 22103 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.113.60:37157: Network error: Client connection negotiation failed: client connection to 127.20.113.60:37157: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.113.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.60.keytab.
Entry for principal kudu/127.20.113.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.60.keytab.
I20260806 09:34:39.719758 20932 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
/tmp/dist-test-taskKR4MEC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786008495763054-20932-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786008495763054-20932-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786008495763054-20932-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786008495763054-20932-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.113.60:37157
--webserver_interface=127.20.113.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.60.keytab
--principal=kudu/127.20.113.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.113.20:33177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.113.62:32933,127.20.113.61:45653,127.20.113.60:37157
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786008495763054-20932-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786008495763054-20932-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786008495763054-20932-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 09:34:39.738142 22103 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.113.60" port: 37157 } attempt: 1
I20260806 09:34:39.739362 22108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:39.738906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.60:37157 (local address 127.0.0.1:36724)
0806 09:34:39.739103 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0806 09:34:39.739209 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.60:37157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260806 09:34:39.740432 22108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:39.740020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.60:37157 (local address 127.0.0.1:36734)
0806 09:34:39.740193 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0806 09:34:39.740264 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.60:37157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260806 09:34:39.740640 22103 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.113.60:37157: Network error: Client connection negotiation failed: client connection to 127.20.113.60:37157: connect: Connection refused (error 111)
I20260806 09:34:39.794636 21994 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.113.61" port: 45653 } attempt: 5
Aug 06 09:34:39 dist-test-slave-45dx krb5kdc[21873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008879, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.62@KRBTEST.COM for kudu/127.20.113.61@KRBTEST.COM
I20260806 09:34:39.808876 22103 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.113.60" port: 37157 } attempt: 2
I20260806 09:34:39.810055 22108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:39.809500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.60:37157 (local address 127.0.0.1:36750)
0806 09:34:39.809741 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0806 09:34:39.809852 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.60:37157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260806 09:34:39.810482 21997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:39.795579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.61:45653 (local address 127.0.0.1:41238)
0806 09:34:39.795846 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0806 09:34:39.795866 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 09:34:39.795988 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:34:39.796422 (+ 434us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:34:39.796426 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:34:39.796842 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:34:39.797037 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:34:39.797047 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:34:39.798539 (+ 1492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:34:39.798544 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:34:39.799410 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:34:39.799423 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:34:39.800477 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:34:39.800481 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:34:39.800666 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:34:39.801366 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:34:39.801392 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:34:39.803538 (+ 2146us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:34:39.807486 (+ 3948us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:34:39.807490 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:34:39.807492 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:34:39.807898 (+ 406us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:34:39.808409 (+ 511us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:34:39.808412 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:34:39.808414 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:34:39.808550 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:34:39.809277 (+ 727us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:34:39.809282 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:34:39.809489 (+ 207us) client_negotiation.cc:769] Sending connection context
0806 09:34:39.809650 (+ 161us) client_negotiation.cc:240] Negotiation successful
0806 09:34:39.809980 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92}
I20260806 09:34:39.810613 22111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:39.795688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41238 (local address 127.20.113.61:45653)
0806 09:34:39.795965 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0806 09:34:39.795974 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 09:34:39.796002 (+ 28us) server_negotiation.cc:407] Connection header received
0806 09:34:39.796115 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:34:39.796120 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:34:39.796210 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:34:39.796367 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:34:39.797223 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:39.798380 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:39.799581 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:39.800293 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:39.800459 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:34:39.803757 (+ 3298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:34:39.803764 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:34:39.803768 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:34:39.803811 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:34:39.807331 (+ 3520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:34:39.808050 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:34:39.808055 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:34:39.808059 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:34:39.808232 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:34:39.808662 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:34:39.808667 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:34:39.808670 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:34:39.809119 (+ 449us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:34:39.809270 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0806 09:34:39.809838 (+ 568us) server_negotiation.cc:299] Negotiation successful
0806 09:34:39.810085 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114}
I20260806 09:34:39.811188 22108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:39.810745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.60:37157 (local address 127.0.0.1:36758)
0806 09:34:39.810928 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0806 09:34:39.811021 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.60:37157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
W20260806 09:34:39.811492 22103 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.113.60:37157: Network error: Client connection negotiation failed: client connection to 127.20.113.60:37157: connect: Connection refused (error 111)
I20260806 09:34:39.812427 21994 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.113.60" port: 37157 } has no permanent_uuid. Determining permanent_uuid...
I20260806 09:34:39.813462 21997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:39.813084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.60:37157 (local address 127.0.0.1:36764)
0806 09:34:39.813243 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0806 09:34:39.813317 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.60:37157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260806 09:34:39.814441 21997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:39.814127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.60:37157 (local address 127.0.0.1:36770)
0806 09:34:39.814257 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0806 09:34:39.814329 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.60:37157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260806 09:34:39.814690 21994 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.113.60:37157: Network error: Client connection negotiation failed: client connection to 127.20.113.60:37157: connect: Connection refused (error 111)
I20260806 09:34:39.865903 21994 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.113.60" port: 37157 } attempt: 1
I20260806 09:34:39.867069 21997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:39.866594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.60:37157 (local address 127.0.0.1:36776)
0806 09:34:39.866784 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0806 09:34:39.866874 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.60:37157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260806 09:34:39.868103 21997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:39.867806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.60:37157 (local address 127.0.0.1:36786)
0806 09:34:39.867936 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0806 09:34:39.867999 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.60:37157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260806 09:34:39.868398 21994 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.113.60:37157: Network error: Client connection negotiation failed: client connection to 127.20.113.60:37157: connect: Connection refused (error 111)
I20260806 09:34:39.902742 22103 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.113.60" port: 37157 } attempt: 3
I20260806 09:34:39.904052 22108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:39.903495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.60:37157 (local address 127.0.0.1:36798)
0806 09:34:39.903768 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0806 09:34:39.903876 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.60:37157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260806 09:34:39.905189 22108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:39.904783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.60:37157 (local address 127.0.0.1:36804)
0806 09:34:39.905021 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0806 09:34:39.905090 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.60:37157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133}
W20260806 09:34:39.905411 22103 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.113.60:37157: Network error: Client connection negotiation failed: client connection to 127.20.113.60:37157: connect: Connection refused (error 111)
I20260806 09:34:39.936609 21994 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.113.60" port: 37157 } attempt: 2
I20260806 09:34:39.937820 21997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:39.937337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.60:37157 (local address 127.0.0.1:36808)
0806 09:34:39.937552 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0806 09:34:39.937645 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.60:37157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260806 09:34:39.938763 21997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:39.938442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.60:37157 (local address 127.0.0.1:36810)
0806 09:34:39.938589 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0806 09:34:39.938654 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.60:37157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260806 09:34:39.938962 21994 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.113.60:37157: Network error: Client connection negotiation failed: client connection to 127.20.113.60:37157: connect: Connection refused (error 111)
I20260806 09:34:40.045171 21994 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.113.60" port: 37157 } attempt: 3
I20260806 09:34:40.046334 21997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:40.045849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.60:37157 (local address 127.0.0.1:36826)
0806 09:34:40.046070 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0806 09:34:40.046164 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.60:37157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260806 09:34:40.047348 21997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:40.046975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.60:37157 (local address 127.0.0.1:36828)
0806 09:34:40.047190 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0806 09:34:40.047252 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.60:37157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
W20260806 09:34:40.047536 21994 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.113.60:37157: Network error: Client connection negotiation failed: client connection to 127.20.113.60:37157: connect: Connection refused (error 111)
I20260806 09:34:40.048597 22103 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.113.60" port: 37157 } attempt: 4
I20260806 09:34:40.049525 22108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:40.049110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.60:37157 (local address 127.0.0.1:36840)
0806 09:34:40.049309 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0806 09:34:40.049383 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.60:37157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260806 09:34:40.050559 22108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:40.050252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.60:37157 (local address 127.0.0.1:36850)
0806 09:34:40.050393 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0806 09:34:40.050462 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.113.60:37157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260806 09:34:40.050797 22103 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.113.60:37157: Network error: Client connection negotiation failed: client connection to 127.20.113.60:37157: connect: Connection refused (error 111)
W20260806 09:34:40.089823 22114 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 09:34:40.090173 22114 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 09:34:40.090298 22114 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 09:34:40.090432 22114 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 09:34:40.097718 22114 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 09:34:40.097839 22114 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 09:34:40.097968 22114 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 09:34:40.098050 22114 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 09:34:40.098141 22114 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 09:34:40.107590 22114 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.113.20:33177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786008495763054-20932-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786008495763054-20932-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.20.113.62:32933,127.20.113.61:45653,127.20.113.60:37157
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786008495763054-20932-0/krb5kdc/kudu_127.20.113.60.keytab
--principal=kudu/127.20.113.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.113.60:37157
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786008495763054-20932-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.20.113.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786008495763054-20932-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 07:10:43 UTC on ccycloud.quasar-ubmxwc.root.comops.site
build id 4569745
ASAN enabled
I20260806 09:34:40.108929 22114 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 09:34:40.110225 22114 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 09:34:40.122207 22121 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:34:40.122067 22123 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 09:34:40.122269 22127 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 09:34:40.123467 22114 server_base.cc:1034] running on GCE node
I20260806 09:34:40.124270 22114 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 09:34:40.125677 22114 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 09:34:40.126930 22114 hybrid_clock.cc:648] HybridClock initialized: now 1786008880126861 us; error 60 us; skew 500 ppm
I20260806 09:34:40.127256 22114 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 09:34:40 dist-test-slave-45dx krb5kdc[21873](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008880, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 09:34:40.130683 22114 init.cc:377] Logged in from keytab as kudu/127.20.113.60@KRBTEST.COM (short username kudu)
I20260806 09:34:40.132095 22114 webserver.cc:460] Webserver started at http://127.20.113.60:37897/ using document root <none> and password file <none>
I20260806 09:34:40.132859 22114 fs_manager.cc:359] Metadata directory not provided
I20260806 09:34:40.132953 22114 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 09:34:40.133227 22114 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 09:34:40.136096 22114 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786008495763054-20932-0/minicluster-data/master-2/data/instance:
uuid: "9eb948dc682344bb8c21303a508482c2"
format_stamp: "Formatted at 2026-08-06 09:34:40 on dist-test-slave-45dx"
server_key: "7aa40545b02b4eef9ed4704b8a7e6962"
server_key_iv: "d4696aafb43b07317ce0dfa0cd869cb2"
server_key_version: "encryptionkey@0"
I20260806 09:34:40.136920 22114 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786008495763054-20932-0/minicluster-data/master-2/wal/instance:
uuid: "9eb948dc682344bb8c21303a508482c2"
format_stamp: "Formatted at 2026-08-06 09:34:40 on dist-test-slave-45dx"
server_key: "7aa40545b02b4eef9ed4704b8a7e6962"
server_key_iv: "d4696aafb43b07317ce0dfa0cd869cb2"
server_key_version: "encryptionkey@0"
I20260806 09:34:40.142583 22114 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260806 09:34:40.145979 22135 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 09:34:40.147291 22114 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 09:34:40.147465 22114 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786008495763054-20932-0/minicluster-data/master-2/data,/tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786008495763054-20932-0/minicluster-data/master-2/wal
uuid: "9eb948dc682344bb8c21303a508482c2"
format_stamp: "Formatted at 2026-08-06 09:34:40 on dist-test-slave-45dx"
server_key: "7aa40545b02b4eef9ed4704b8a7e6962"
server_key_iv: "d4696aafb43b07317ce0dfa0cd869cb2"
server_key_version: "encryptionkey@0"
I20260806 09:34:40.147699 22114 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786008495763054-20932-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786008495763054-20932-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786008495763054-20932-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 09:34:40.162787 22114 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 09:34:40.167119 22114 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 09:34:40.167402 22114 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 09:34:40.186467 22114 rpc_server.cc:307] RPC server started. Bound to: 127.20.113.60:37157
I20260806 09:34:40.186486 22210 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.113.60:37157 every 8 connection(s)
I20260806 09:34:40.187788 22114 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786008495763054-20932-0/minicluster-data/master-2/data/info.pb
I20260806 09:34:40.188140 20932 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu as pid 22114
I20260806 09:34:40.188390 20932 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKR4MEC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786008495763054-20932-0/minicluster-data/master-2/wal/instance
I20260806 09:34:40.189043 20932 external_mini_cluster.cc:1428] Setting key 508e2f6f9a0164c5b4fe5a61a0544348
I20260806 09:34:40.192859 22211 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 09:34:40.201475 22211 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.113.62" port: 32933 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 09:34:40 dist-test-slave-45dx krb5kdc[21873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008878, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.113.60@KRBTEST.COM
Aug 06 09:34:40 dist-test-slave-45dx krb5kdc[21873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008880, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.60@KRBTEST.COM for kudu/127.20.113.62@KRBTEST.COM
I20260806 09:34:40.213374 22215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:40.190876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36864 (local address 127.20.113.60:37157)
0806 09:34:40.191638 (+ 762us) server_negotiation.cc:206] Beginning negotiation
0806 09:34:40.191653 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 09:34:40.191702 (+ 49us) server_negotiation.cc:407] Connection header received
0806 09:34:40.194039 (+ 2337us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:34:40.194068 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:34:40.194545 (+ 477us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:34:40.194958 (+ 413us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:34:40.196280 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:40.198292 (+ 2012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:40.199549 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:40.200564 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:40.200750 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:34:40.205420 (+ 4670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:34:40.205430 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:34:40.205447 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:34:40.205489 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:34:40.208612 (+ 3123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:34:40.209385 (+ 773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:34:40.209393 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:34:40.209401 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:34:40.209613 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:34:40.210119 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:34:40.210123 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:34:40.210125 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:34:40.210820 (+ 695us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:34:40.211047 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0806 09:34:40.211536 (+ 489us) server_negotiation.cc:299] Negotiation successful
0806 09:34:40.212103 (+ 567us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":536,"thread_start_us":312,"threads_started":1}
I20260806 09:34:40.221263 22217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:40.203797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.62:32933 (local address 127.0.0.1:51150)
0806 09:34:40.204583 (+ 786us) negotiation.cc:107] Waiting for socket to connect
0806 09:34:40.204642 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0806 09:34:40.204787 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:34:40.205615 (+ 828us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:34:40.205621 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:34:40.206357 (+ 736us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:34:40.206641 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:34:40.206651 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:34:40.207677 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:34:40.207681 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:34:40.208849 (+ 1168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:34:40.208868 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:34:40.209800 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:34:40.209807 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:34:40.210002 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:34:40.210683 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:34:40.210701 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:34:40.213382 (+ 2681us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:34:40.218679 (+ 5297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:34:40.218689 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:34:40.218693 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:34:40.219073 (+ 380us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:34:40.219473 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:34:40.219479 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:34:40.219482 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:34:40.219606 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:34:40.220111 (+ 505us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:34:40.220119 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:34:40.220275 (+ 156us) client_negotiation.cc:769] Sending connection context
0806 09:34:40.220538 (+ 263us) client_negotiation.cc:240] Negotiation successful
0806 09:34:40.220829 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":520,"thread_start_us":312,"threads_started":1}
I20260806 09:34:40.221329 22219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:40.203919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51150 (local address 127.20.113.62:32933)
0806 09:34:40.205301 (+ 1382us) server_negotiation.cc:206] Beginning negotiation
0806 09:34:40.205307 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:34:40.205326 (+ 19us) server_negotiation.cc:407] Connection header received
0806 09:34:40.205375 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:34:40.205378 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:34:40.205438 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:34:40.205561 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:34:40.206814 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:40.207511 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:40.209019 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:40.209560 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:40.209709 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:34:40.214522 (+ 4813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:34:40.214527 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:34:40.214530 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:34:40.214564 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:34:40.218507 (+ 3943us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:34:40.219209 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:34:40.219218 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:34:40.219220 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:34:40.219355 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:34:40.219738 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:34:40.219742 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:34:40.219744 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:34:40.219979 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:34:40.220126 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0806 09:34:40.220692 (+ 566us) server_negotiation.cc:299] Negotiation successful
0806 09:34:40.220916 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1255,"thread_start_us":1062,"threads_started":1}
I20260806 09:34:40.222669 22211 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.113.61" port: 45653 } has no permanent_uuid. Determining permanent_uuid...
I20260806 09:34:40.224735 21994 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.113.60" port: 37157 } attempt: 4
Aug 06 09:34:40 dist-test-slave-45dx krb5kdc[21873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008878, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.113.62@KRBTEST.COM
Aug 06 09:34:40 dist-test-slave-45dx krb5kdc[21873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008880, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.60@KRBTEST.COM for kudu/127.20.113.61@KRBTEST.COM
Aug 06 09:34:40 dist-test-slave-45dx krb5kdc[21873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008879, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.62@KRBTEST.COM for kudu/127.20.113.60@KRBTEST.COM
I20260806 09:34:40.236531 22221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:40.216183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51160 (local address 127.20.113.62:32933)
0806 09:34:40.219046 (+ 2863us) server_negotiation.cc:206] Beginning negotiation
0806 09:34:40.219049 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 09:34:40.219066 (+ 17us) server_negotiation.cc:407] Connection header received
0806 09:34:40.219101 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:34:40.219103 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:34:40.219149 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:34:40.219264 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:34:40.220453 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:40.221281 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:40.222841 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:40.223350 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:40.223483 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:34:40.228794 (+ 5311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:34:40.228799 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:34:40.228802 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:34:40.228839 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:34:40.232491 (+ 3652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:34:40.233379 (+ 888us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:34:40.233384 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:34:40.233389 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:34:40.233582 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:34:40.234104 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:34:40.234109 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:34:40.234115 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:34:40.234448 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:34:40.234718 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0806 09:34:40.235859 (+ 1141us) server_negotiation.cc:299] Negotiation successful
0806 09:34:40.236077 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2765,"thread_start_us":2628,"threads_started":1}
I20260806 09:34:40.239852 21997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:40.225259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.60:37157 (local address 127.0.0.1:36876)
0806 09:34:40.225468 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0806 09:34:40.225485 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 09:34:40.225586 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:34:40.225915 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:34:40.225921 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:34:40.226237 (+ 316us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:34:40.226370 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:34:40.226377 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:34:40.227648 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:34:40.227654 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:34:40.229056 (+ 1402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:34:40.229070 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:34:40.230149 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:34:40.230155 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:34:40.230397 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:34:40.231087 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:34:40.231117 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:34:40.233821 (+ 2704us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:34:40.236977 (+ 3156us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:34:40.236984 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:34:40.236991 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:34:40.237530 (+ 539us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:34:40.237917 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:34:40.237923 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:34:40.237927 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:34:40.238059 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:34:40.238891 (+ 832us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:34:40.238897 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:34:40.239068 (+ 171us) client_negotiation.cc:769] Sending connection context
0806 09:34:40.239229 (+ 161us) client_negotiation.cc:240] Negotiation successful
0806 09:34:40.239488 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116}
I20260806 09:34:40.240151 22217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:40.223453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.61:45653 (local address 127.0.0.1:41242)
0806 09:34:40.223708 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0806 09:34:40.223722 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 09:34:40.223791 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:34:40.224119 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:34:40.224123 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:34:40.224515 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:34:40.224641 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:34:40.224648 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:34:40.226505 (+ 1857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:34:40.226509 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:34:40.227880 (+ 1371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:34:40.227895 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:34:40.229023 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:34:40.229029 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:34:40.229249 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:34:40.229942 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:34:40.229958 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:34:40.232540 (+ 2582us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:34:40.236873 (+ 4333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:34:40.236881 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:34:40.236886 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:34:40.237399 (+ 513us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:34:40.238193 (+ 794us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:34:40.238197 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:34:40.238200 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:34:40.238430 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:34:40.239116 (+ 686us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:34:40.239121 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:34:40.239274 (+ 153us) client_negotiation.cc:769] Sending connection context
0806 09:34:40.239423 (+ 149us) client_negotiation.cc:240] Negotiation successful
0806 09:34:40.239712 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"mutex_wait_us":74}
I20260806 09:34:40.240191 22215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:40.225342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36876 (local address 127.20.113.60:37157)
0806 09:34:40.225477 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0806 09:34:40.225481 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 09:34:40.225563 (+ 82us) server_negotiation.cc:407] Connection header received
0806 09:34:40.225683 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:34:40.225689 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:34:40.225765 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:34:40.225868 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:34:40.226506 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:40.227517 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:40.229206 (+ 1689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:40.229980 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:40.230113 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:34:40.234034 (+ 3921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:34:40.234038 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:34:40.234041 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:34:40.234069 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:34:40.236815 (+ 2746us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:34:40.237619 (+ 804us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:34:40.237628 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:34:40.237631 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:34:40.237781 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:34:40.238172 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:34:40.238177 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:34:40.238180 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:34:40.238557 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:34:40.238718 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0806 09:34:40.239329 (+ 611us) server_negotiation.cc:299] Negotiation successful
0806 09:34:40.239548 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":77,"server-negotiator.queue_time_us":40}
I20260806 09:34:40.241195 22111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:40.223556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41242 (local address 127.20.113.61:45653)
0806 09:34:40.223741 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0806 09:34:40.223745 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 09:34:40.223761 (+ 16us) server_negotiation.cc:407] Connection header received
0806 09:34:40.223921 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:34:40.223927 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:34:40.223997 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:34:40.224096 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:34:40.224777 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:40.225785 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:40.228034 (+ 2249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:40.228679 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:40.228800 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:34:40.232749 (+ 3949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:34:40.232758 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:34:40.232766 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:34:40.232811 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:34:40.236479 (+ 3668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:34:40.237870 (+ 1391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:34:40.237876 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:34:40.237879 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:34:40.238053 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:34:40.238543 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:34:40.238549 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:34:40.238552 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:34:40.238964 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:34:40.240446 (+ 1482us) server_negotiation.cc:1035] Waiting for connection context
0806 09:34:40.240589 (+ 143us) server_negotiation.cc:299] Negotiation successful
0806 09:34:40.240794 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 09:34:40.242308 22211 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.113.60" port: 37157 } has no permanent_uuid. Determining permanent_uuid...
I20260806 09:34:40.246326 21994 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d5cfae47be6541ef8b7717525d447fd7: Bootstrap starting.
Aug 06 09:34:40 dist-test-slave-45dx krb5kdc[21873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008878, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.113.61@KRBTEST.COM
I20260806 09:34:40.249730 21994 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d5cfae47be6541ef8b7717525d447fd7: Neither blocks nor log segments found. Creating new log.
Aug 06 09:34:40 dist-test-slave-45dx krb5kdc[21873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008880, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.60@KRBTEST.COM for kudu/127.20.113.60@KRBTEST.COM
I20260806 09:34:40.251078 21994 log.cc:824] T 00000000000000000000000000000000 P d5cfae47be6541ef8b7717525d447fd7: Log is configured to *not* fsync() on all Append() calls
I20260806 09:34:40.256371 22223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:40.238453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41258 (local address 127.20.113.61:45653)
0806 09:34:40.239041 (+ 588us) server_negotiation.cc:206] Beginning negotiation
0806 09:34:40.239048 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:34:40.239071 (+ 23us) server_negotiation.cc:407] Connection header received
0806 09:34:40.239115 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:34:40.239119 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:34:40.239182 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:34:40.239273 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:34:40.241127 (+ 1854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:40.242392 (+ 1265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:40.244357 (+ 1965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:40.244885 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:40.245017 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:34:40.249835 (+ 4818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:34:40.249843 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:34:40.249851 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:34:40.249893 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:34:40.253202 (+ 3309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:34:40.254055 (+ 853us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:34:40.254061 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:34:40.254065 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:34:40.254226 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:34:40.254702 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:34:40.254709 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:34:40.254715 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:34:40.255056 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:34:40.255331 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0806 09:34:40.255677 (+ 346us) server_negotiation.cc:299] Negotiation successful
0806 09:34:40.255912 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":454,"thread_start_us":272,"threads_started":1}
I20260806 09:34:40.256606 21994 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d5cfae47be6541ef8b7717525d447fd7: No bootstrap required, opened a new log
I20260806 09:34:40.258215 22217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:40.243262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.60:37157 (local address 127.0.0.1:36882)
0806 09:34:40.243486 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0806 09:34:40.243500 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 09:34:40.243630 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:34:40.243920 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:34:40.243924 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:34:40.244298 (+ 374us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:34:40.244488 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:34:40.244498 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:34:40.245734 (+ 1236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:34:40.245738 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:34:40.247127 (+ 1389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:34:40.247143 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:34:40.248086 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:34:40.248092 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:34:40.248333 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:34:40.248855 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:34:40.248878 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:34:40.251616 (+ 2738us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:34:40.255377 (+ 3761us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:34:40.255381 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:34:40.255384 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:34:40.255879 (+ 495us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:34:40.256280 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:34:40.256284 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:34:40.256288 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:34:40.256498 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:34:40.257202 (+ 704us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:34:40.257208 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:34:40.257388 (+ 180us) client_negotiation.cc:769] Sending connection context
0806 09:34:40.257552 (+ 164us) client_negotiation.cc:240] Negotiation successful
0806 09:34:40.257848 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141}
I20260806 09:34:40.259050 22215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:40.243229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36882 (local address 127.20.113.60:37157)
0806 09:34:40.243426 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0806 09:34:40.243432 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:34:40.243570 (+ 138us) server_negotiation.cc:407] Connection header received
0806 09:34:40.243734 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:34:40.243738 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:34:40.243804 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:34:40.243903 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:34:40.244634 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:40.245601 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:40.247292 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:40.247901 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:40.248031 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:34:40.251886 (+ 3855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:34:40.251896 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:34:40.251903 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:34:40.251949 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:34:40.255246 (+ 3297us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:34:40.256019 (+ 773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:34:40.256023 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:34:40.256026 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:34:40.256168 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:34:40.256616 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:34:40.256620 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:34:40.256623 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:34:40.257052 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:34:40.257235 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0806 09:34:40.258524 (+ 1289us) server_negotiation.cc:299] Negotiation successful
0806 09:34:40.258736 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 09:34:40.261648 20932 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 09:34:40.264776 22211 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9eb948dc682344bb8c21303a508482c2: Bootstrap starting.
I20260806 09:34:40.265942 21959 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "d5cfae47be6541ef8b7717525d447fd7"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51160
I20260806 09:34:40.267635 21994 raft_consensus.cc:348] T 00000000000000000000000000000000 P d5cfae47be6541ef8b7717525d447fd7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5cfae47be6541ef8b7717525d447fd7" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 32933 } } peers { permanent_uuid: "18af5fcd3b6747348add713a9330a7b4" member_type: VOTER last_known_addr { host: "127.20.113.61" port: 45653 } } peers { permanent_uuid: "9eb948dc682344bb8c21303a508482c2" member_type: VOTER last_known_addr { host: "127.20.113.60" port: 37157 } }
I20260806 09:34:40.268038 21994 raft_consensus.cc:729] T 00000000000000000000000000000000 P d5cfae47be6541ef8b7717525d447fd7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5cfae47be6541ef8b7717525d447fd7, State: Initialized, Role: FOLLOWER
I20260806 09:34:40.268626 22211 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9eb948dc682344bb8c21303a508482c2: Neither blocks nor log segments found. Creating new log.
I20260806 09:34:40.268548 21994 consensus_queue.cc:260] T 00000000000000000000000000000000 P d5cfae47be6541ef8b7717525d447fd7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5cfae47be6541ef8b7717525d447fd7" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 32933 } } peers { permanent_uuid: "18af5fcd3b6747348add713a9330a7b4" member_type: VOTER last_known_addr { host: "127.20.113.61" port: 45653 } } peers { permanent_uuid: "9eb948dc682344bb8c21303a508482c2" member_type: VOTER last_known_addr { host: "127.20.113.60" port: 37157 } }
I20260806 09:34:40.269966 22211 log.cc:824] T 00000000000000000000000000000000 P 9eb948dc682344bb8c21303a508482c2: Log is configured to *not* fsync() on all Append() calls
I20260806 09:34:40.271445 22228 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5cfae47be6541ef8b7717525d447fd7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5cfae47be6541ef8b7717525d447fd7" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 32933 } } peers { permanent_uuid: "18af5fcd3b6747348add713a9330a7b4" member_type: VOTER last_known_addr { host: "127.20.113.61" port: 45653 } } peers { permanent_uuid: "9eb948dc682344bb8c21303a508482c2" member_type: VOTER last_known_addr { host: "127.20.113.60" port: 37157 } } }
I20260806 09:34:40.271860 22228 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5cfae47be6541ef8b7717525d447fd7 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 09:34:40.272802 21994 sys_catalog.cc:564] T 00000000000000000000000000000000 P d5cfae47be6541ef8b7717525d447fd7 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 09:34:40.272891 21959 raft_consensus.cc:482] T 00000000000000000000000000000000 P d5cfae47be6541ef8b7717525d447fd7 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260806 09:34:40.273092 21959 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d5cfae47be6541ef8b7717525d447fd7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:34:40.273646 22211 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9eb948dc682344bb8c21303a508482c2: No bootstrap required, opened a new log
I20260806 09:34:40.277280 21959 raft_consensus.cc:504] T 00000000000000000000000000000000 P d5cfae47be6541ef8b7717525d447fd7 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5cfae47be6541ef8b7717525d447fd7" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 32933 } } peers { permanent_uuid: "18af5fcd3b6747348add713a9330a7b4" member_type: VOTER last_known_addr { host: "127.20.113.61" port: 45653 } } peers { permanent_uuid: "9eb948dc682344bb8c21303a508482c2" member_type: VOTER last_known_addr { host: "127.20.113.60" port: 37157 } }
I20260806 09:34:40.280359 21959 leader_election.cc:288] T 00000000000000000000000000000000 P d5cfae47be6541ef8b7717525d447fd7 [CANDIDATE]: Term 1 election: Requested vote from peers 18af5fcd3b6747348add713a9330a7b4 (127.20.113.61:45653), 9eb948dc682344bb8c21303a508482c2 (127.20.113.60:37157)
I20260806 09:34:40.285395 22211 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9eb948dc682344bb8c21303a508482c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5cfae47be6541ef8b7717525d447fd7" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 32933 } } peers { permanent_uuid: "18af5fcd3b6747348add713a9330a7b4" member_type: VOTER last_known_addr { host: "127.20.113.61" port: 45653 } } peers { permanent_uuid: "9eb948dc682344bb8c21303a508482c2" member_type: VOTER last_known_addr { host: "127.20.113.60" port: 37157 } }
I20260806 09:34:40.285861 22211 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9eb948dc682344bb8c21303a508482c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9eb948dc682344bb8c21303a508482c2, State: Initialized, Role: FOLLOWER
I20260806 09:34:40.286397 22211 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9eb948dc682344bb8c21303a508482c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5cfae47be6541ef8b7717525d447fd7" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 32933 } } peers { permanent_uuid: "18af5fcd3b6747348add713a9330a7b4" member_type: VOTER last_known_addr { host: "127.20.113.61" port: 45653 } } peers { permanent_uuid: "9eb948dc682344bb8c21303a508482c2" member_type: VOTER last_known_addr { host: "127.20.113.60" port: 37157 } }
I20260806 09:34:40.310506 22221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:40.287948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51166 (local address 127.20.113.62:32933)
0806 09:34:40.288564 (+ 616us) server_negotiation.cc:206] Beginning negotiation
0806 09:34:40.288573 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 09:34:40.288598 (+ 25us) server_negotiation.cc:407] Connection header received
0806 09:34:40.288688 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:34:40.288696 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:34:40.288797 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:34:40.288939 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:34:40.291842 (+ 2903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:40.293369 (+ 1527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:40.295874 (+ 2505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:40.296853 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:40.296972 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:34:40.301481 (+ 4509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:34:40.301491 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:34:40.301498 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:34:40.301548 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:34:40.306240 (+ 4692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:34:40.307344 (+ 1104us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:34:40.307353 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:34:40.307358 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:34:40.307557 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:34:40.308100 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:34:40.308107 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:34:40.308113 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:34:40.308598 (+ 485us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:34:40.308907 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0806 09:34:40.309685 (+ 778us) server_negotiation.cc:299] Negotiation successful
0806 09:34:40.309983 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":419}
W20260806 09:34:40.322770 21926 tablet.cc:2369] T 00000000000000000000000000000000 P d5cfae47be6541ef8b7717525d447fd7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 09:34:40.329330 22173 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d5cfae47be6541ef8b7717525d447fd7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "9eb948dc682344bb8c21303a508482c2"
I20260806 09:34:40.329197 22243 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9eb948dc682344bb8c21303a508482c2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5cfae47be6541ef8b7717525d447fd7" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 32933 } } peers { permanent_uuid: "18af5fcd3b6747348add713a9330a7b4" member_type: VOTER last_known_addr { host: "127.20.113.61" port: 45653 } } peers { permanent_uuid: "9eb948dc682344bb8c21303a508482c2" member_type: VOTER last_known_addr { host: "127.20.113.60" port: 37157 } } }
I20260806 09:34:40.329949 22173 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9eb948dc682344bb8c21303a508482c2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 09:34:40.330139 22243 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9eb948dc682344bb8c21303a508482c2 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 09:34:40.333711 22173 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9eb948dc682344bb8c21303a508482c2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d5cfae47be6541ef8b7717525d447fd7 in term 1.
I20260806 09:34:40.334434 21921 leader_election.cc:302] T 00000000000000000000000000000000 P d5cfae47be6541ef8b7717525d447fd7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9eb948dc682344bb8c21303a508482c2, d5cfae47be6541ef8b7717525d447fd7; no voters:
I20260806 09:34:40.329308 22211 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9eb948dc682344bb8c21303a508482c2 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 09:34:40.334954 22228 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d5cfae47be6541ef8b7717525d447fd7 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 09:34:40.337963 22253 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 09:34:40.340986 22215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:40.286937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36888 (local address 127.20.113.60:37157)
0806 09:34:40.287122 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0806 09:34:40.287129 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:34:40.311720 (+ 24591us) server_negotiation.cc:407] Connection header received
0806 09:34:40.312046 (+ 326us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:34:40.312055 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:34:40.312165 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:34:40.312250 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:34:40.314632 (+ 2382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:40.315967 (+ 1335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:40.319299 (+ 3332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:40.320183 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:40.320373 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:34:40.331970 (+ 11597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:34:40.331980 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:34:40.331988 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:34:40.332048 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:34:40.337085 (+ 5037us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:34:40.338163 (+ 1078us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:34:40.338171 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:34:40.338177 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:34:40.338346 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:34:40.338852 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:34:40.338857 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:34:40.338860 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:34:40.339242 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:34:40.339510 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0806 09:34:40.340192 (+ 682us) server_negotiation.cc:299] Negotiation successful
0806 09:34:40.340599 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 09:34:40.347407 22228 raft_consensus.cc:686] T 00000000000000000000000000000000 P d5cfae47be6541ef8b7717525d447fd7 [term 1 LEADER]: Becoming Leader. State: Replica: d5cfae47be6541ef8b7717525d447fd7, State: Running, Role: LEADER
I20260806 09:34:40.348274 22228 consensus_queue.cc:237] T 00000000000000000000000000000000 P d5cfae47be6541ef8b7717525d447fd7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5cfae47be6541ef8b7717525d447fd7" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 32933 } } peers { permanent_uuid: "18af5fcd3b6747348add713a9330a7b4" member_type: VOTER last_known_addr { host: "127.20.113.61" port: 45653 } } peers { permanent_uuid: "9eb948dc682344bb8c21303a508482c2" member_type: VOTER last_known_addr { host: "127.20.113.60" port: 37157 } }
I20260806 09:34:40.349979 22103 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.113.60" port: 37157 } attempt: 5
I20260806 09:34:40.355681 22223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:40.288215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41272 (local address 127.20.113.61:45653)
0806 09:34:40.288529 (+ 314us) server_negotiation.cc:206] Beginning negotiation
0806 09:34:40.288538 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 09:34:40.324590 (+ 36052us) server_negotiation.cc:407] Connection header received
0806 09:34:40.325324 (+ 734us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:34:40.325333 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:34:40.325449 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:34:40.325571 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:34:40.331049 (+ 5478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:40.332123 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:40.343200 (+ 11077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:40.343893 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:40.344030 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:34:40.347218 (+ 3188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:34:40.347223 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:34:40.347226 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:34:40.347261 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:34:40.351088 (+ 3827us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:34:40.351927 (+ 839us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:34:40.351934 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:34:40.351940 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:34:40.352093 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:34:40.352711 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:34:40.352715 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:34:40.352718 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:34:40.353257 (+ 539us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:34:40.354566 (+ 1309us) server_negotiation.cc:1035] Waiting for connection context
0806 09:34:40.354702 (+ 136us) server_negotiation.cc:299] Negotiation successful
0806 09:34:40.354951 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126}
I20260806 09:34:40.359418 22254 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5cfae47be6541ef8b7717525d447fd7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d5cfae47be6541ef8b7717525d447fd7. Latest consensus state: current_term: 1 leader_uuid: "d5cfae47be6541ef8b7717525d447fd7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5cfae47be6541ef8b7717525d447fd7" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 32933 } } peers { permanent_uuid: "18af5fcd3b6747348add713a9330a7b4" member_type: VOTER last_known_addr { host: "127.20.113.61" port: 45653 } } peers { permanent_uuid: "9eb948dc682344bb8c21303a508482c2" member_type: VOTER last_known_addr { host: "127.20.113.60" port: 37157 } } }
I20260806 09:34:40.360054 22254 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5cfae47be6541ef8b7717525d447fd7 [sys.catalog]: This master's current role is: LEADER
Aug 06 09:34:40 dist-test-slave-45dx krb5kdc[21873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786008879, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.113.61@KRBTEST.COM for kudu/127.20.113.60@KRBTEST.COM
I20260806 09:34:40.366781 22260 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 09:34:40.368546 22108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:40.350543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.113.60:37157 (local address 127.0.0.1:36890)
0806 09:34:40.350932 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0806 09:34:40.350952 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 09:34:40.351075 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 09:34:40.352096 (+ 1021us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 09:34:40.352101 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 09:34:40.352658 (+ 557us) client_negotiation.cc:189] Negotiated authn=SASL
0806 09:34:40.353030 (+ 372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:34:40.353045 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:34:40.355555 (+ 2510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:34:40.355560 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:34:40.356795 (+ 1235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 09:34:40.356808 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 09:34:40.357902 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 09:34:40.357908 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 09:34:40.358106 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:34:40.358659 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 09:34:40.358681 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 09:34:40.361476 (+ 2795us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 09:34:40.365793 (+ 4317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:34:40.365799 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:34:40.365803 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:34:40.366295 (+ 492us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:34:40.366732 (+ 437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 09:34:40.366740 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 09:34:40.366744 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 09:34:40.366910 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 09:34:40.367580 (+ 670us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 09:34:40.367585 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 09:34:40.367740 (+ 155us) client_negotiation.cc:769] Sending connection context
0806 09:34:40.367887 (+ 147us) client_negotiation.cc:240] Negotiation successful
0806 09:34:40.368160 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250}
I20260806 09:34:40.369601 22215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:40.351297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36890 (local address 127.20.113.60:37157)
0806 09:34:40.351583 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0806 09:34:40.351589 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:34:40.351634 (+ 45us) server_negotiation.cc:407] Connection header received
0806 09:34:40.351713 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:34:40.351718 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:34:40.351885 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:34:40.353220 (+ 1335us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:34:40.353512 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:40.355338 (+ 1826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:40.356935 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:40.357728 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:40.357864 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:34:40.361887 (+ 4023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 09:34:40.361893 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 09:34:40.361898 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 09:34:40.361936 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 09:34:40.365633 (+ 3697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:34:40.366459 (+ 826us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:34:40.366465 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:34:40.366468 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:34:40.366601 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 09:34:40.367058 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 09:34:40.367062 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 09:34:40.367067 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 09:34:40.367435 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 09:34:40.368808 (+ 1373us) server_negotiation.cc:1035] Waiting for connection context
0806 09:34:40.368967 (+ 159us) server_negotiation.cc:299] Negotiation successful
0806 09:34:40.369188 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121}
I20260806 09:34:40.375144 22260 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 09:34:40.375344 22103 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 18af5fcd3b6747348add713a9330a7b4: Bootstrap starting.
I20260806 09:34:40.380908 22103 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 18af5fcd3b6747348add713a9330a7b4: Neither blocks nor log segments found. Creating new log.
I20260806 09:34:40.386790 22103 log.cc:824] T 00000000000000000000000000000000 P 18af5fcd3b6747348add713a9330a7b4: Log is configured to *not* fsync() on all Append() calls
I20260806 09:34:40.359915 21894 master_runner.cc:186] Error getting master registration for 127.20.113.61:45653: OK, instance_id { permanent_uuid: "18af5fcd3b6747348add713a9330a7b4" instance_seqno: 1786008879615032 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 09:34:40.387889 21894 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 09:34:40.390815 22103 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 18af5fcd3b6747348add713a9330a7b4: No bootstrap required, opened a new log
I20260806 09:34:40.395393 22173 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9eb948dc682344bb8c21303a508482c2 [term 1 FOLLOWER]: Refusing update from remote peer d5cfae47be6541ef8b7717525d447fd7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 09:34:40.397092 22228 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d5cfae47be6541ef8b7717525d447fd7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9eb948dc682344bb8c21303a508482c2" member_type: VOTER last_known_addr { host: "127.20.113.60" port: 37157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 09:34:40.405964 22103 raft_consensus.cc:348] T 00000000000000000000000000000000 P 18af5fcd3b6747348add713a9330a7b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5cfae47be6541ef8b7717525d447fd7" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 32933 } } peers { permanent_uuid: "18af5fcd3b6747348add713a9330a7b4" member_type: VOTER last_known_addr { host: "127.20.113.61" port: 45653 } } peers { permanent_uuid: "9eb948dc682344bb8c21303a508482c2" member_type: VOTER last_known_addr { host: "127.20.113.60" port: 37157 } }
I20260806 09:34:40.406574 22103 raft_consensus.cc:729] T 00000000000000000000000000000000 P 18af5fcd3b6747348add713a9330a7b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18af5fcd3b6747348add713a9330a7b4, State: Initialized, Role: FOLLOWER
I20260806 09:34:40.407357 22103 consensus_queue.cc:260] T 00000000000000000000000000000000 P 18af5fcd3b6747348add713a9330a7b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5cfae47be6541ef8b7717525d447fd7" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 32933 } } peers { permanent_uuid: "18af5fcd3b6747348add713a9330a7b4" member_type: VOTER last_known_addr { host: "127.20.113.61" port: 45653 } } peers { permanent_uuid: "9eb948dc682344bb8c21303a508482c2" member_type: VOTER last_known_addr { host: "127.20.113.60" port: 37157 } }
I20260806 09:34:40.418181 22072 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d5cfae47be6541ef8b7717525d447fd7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "18af5fcd3b6747348add713a9330a7b4"
I20260806 09:34:40.419001 22071 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 18af5fcd3b6747348add713a9330a7b4 [term 0 FOLLOWER]: Advancing to term 1
W20260806 09:34:40.419485 22144 tablet.cc:2369] T 00000000000000000000000000000000 P 9eb948dc682344bb8c21303a508482c2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 09:34:40.422636 22071 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 18af5fcd3b6747348add713a9330a7b4 [term 1 FOLLOWER]: Refusing update from remote peer d5cfae47be6541ef8b7717525d447fd7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 09:34:40.423517 22228 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d5cfae47be6541ef8b7717525d447fd7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "18af5fcd3b6747348add713a9330a7b4" member_type: VOTER last_known_addr { host: "127.20.113.61" port: 45653 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 09:34:40.430546 22103 sys_catalog.cc:564] T 00000000000000000000000000000000 P 18af5fcd3b6747348add713a9330a7b4 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 09:34:40.435568 22243 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9eb948dc682344bb8c21303a508482c2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d5cfae47be6541ef8b7717525d447fd7. Latest consensus state: current_term: 1 leader_uuid: "d5cfae47be6541ef8b7717525d447fd7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5cfae47be6541ef8b7717525d447fd7" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 32933 } } peers { permanent_uuid: "18af5fcd3b6747348add713a9330a7b4" member_type: VOTER last_known_addr { host: "127.20.113.61" port: 45653 } } peers { permanent_uuid: "9eb948dc682344bb8c21303a508482c2" member_type: VOTER last_known_addr { host: "127.20.113.60" port: 37157 } } }
I20260806 09:34:40.435963 22243 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9eb948dc682344bb8c21303a508482c2 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 09:34:40.441879 22284 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18af5fcd3b6747348add713a9330a7b4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d5cfae47be6541ef8b7717525d447fd7. Latest consensus state: current_term: 1 leader_uuid: "d5cfae47be6541ef8b7717525d447fd7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5cfae47be6541ef8b7717525d447fd7" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 32933 } } peers { permanent_uuid: "18af5fcd3b6747348add713a9330a7b4" member_type: VOTER last_known_addr { host: "127.20.113.61" port: 45653 } } peers { permanent_uuid: "9eb948dc682344bb8c21303a508482c2" member_type: VOTER last_known_addr { host: "127.20.113.60" port: 37157 } } }
I20260806 09:34:40.442433 22284 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18af5fcd3b6747348add713a9330a7b4 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 09:34:40.442077 22287 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18af5fcd3b6747348add713a9330a7b4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d5cfae47be6541ef8b7717525d447fd7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5cfae47be6541ef8b7717525d447fd7" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 32933 } } peers { permanent_uuid: "18af5fcd3b6747348add713a9330a7b4" member_type: VOTER last_known_addr { host: "127.20.113.61" port: 45653 } } peers { permanent_uuid: "9eb948dc682344bb8c21303a508482c2" member_type: VOTER last_known_addr { host: "127.20.113.60" port: 37157 } } }
I20260806 09:34:40.442960 22287 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18af5fcd3b6747348add713a9330a7b4 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 09:34:40.445636 22276 mvcc.cc:205] Tried to move back new op lower bound from 7315492374069047296 to 7315492373914542080. Current Snapshot: MvccSnapshot[applied={T|T < 7315492374069047296}]
I20260806 09:34:40.448844 22283 mvcc.cc:205] Tried to move back new op lower bound from 7315492374069047296 to 7315492373914542080. Current Snapshot: MvccSnapshot[applied={T|T < 7315492374069047296}]
I20260806 09:34:40.442497 22114 master_runner.cc:186] Error getting master registration for 127.20.113.61:45653: OK, instance_id { permanent_uuid: "18af5fcd3b6747348add713a9330a7b4" instance_seqno: 1786008879615032 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 09:34:40.455772 22114 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 09:34:40.457219 22072 raft_consensus.cc:2396] T 00000000000000000000000000000000 P 18af5fcd3b6747348add713a9330a7b4 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate d5cfae47be6541ef8b7717525d447fd7 for term 1 because replica has last-logged OpId of term: 1 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260806 09:34:40.460052 22288 mvcc.cc:205] Tried to move back new op lower bound from 7315492374069047296 to 7315492373914542080. Current Snapshot: MvccSnapshot[applied={T|T < 7315492374069047296}]
I20260806 09:34:40.461687 22287 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18af5fcd3b6747348add713a9330a7b4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d5cfae47be6541ef8b7717525d447fd7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5cfae47be6541ef8b7717525d447fd7" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 32933 } } peers { permanent_uuid: "18af5fcd3b6747348add713a9330a7b4" member_type: VOTER last_known_addr { host: "127.20.113.61" port: 45653 } } peers { permanent_uuid: "9eb948dc682344bb8c21303a508482c2" member_type: VOTER last_known_addr { host: "127.20.113.60" port: 37157 } } }
I20260806 09:34:40.462044 22287 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18af5fcd3b6747348add713a9330a7b4 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 09:34:40.462824 22243 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9eb948dc682344bb8c21303a508482c2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d5cfae47be6541ef8b7717525d447fd7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5cfae47be6541ef8b7717525d447fd7" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 32933 } } peers { permanent_uuid: "18af5fcd3b6747348add713a9330a7b4" member_type: VOTER last_known_addr { host: "127.20.113.61" port: 45653 } } peers { permanent_uuid: "9eb948dc682344bb8c21303a508482c2" member_type: VOTER last_known_addr { host: "127.20.113.60" port: 37157 } } }
I20260806 09:34:40.463112 22243 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9eb948dc682344bb8c21303a508482c2 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 09:34:40.466168 22260 catalog_manager.cc:1347] Generated new cluster ID: 362185097cca46cfae3831a65a238d31
I20260806 09:34:40.466290 22260 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
W20260806 09:34:40.467707 22280 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9eb948dc682344bb8c21303a508482c2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 09:34:40.467837 22280 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 09:34:40.474550 22254 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5cfae47be6541ef8b7717525d447fd7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d5cfae47be6541ef8b7717525d447fd7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5cfae47be6541ef8b7717525d447fd7" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 32933 } } peers { permanent_uuid: "18af5fcd3b6747348add713a9330a7b4" member_type: VOTER last_known_addr { host: "127.20.113.61" port: 45653 } } peers { permanent_uuid: "9eb948dc682344bb8c21303a508482c2" member_type: VOTER last_known_addr { host: "127.20.113.60" port: 37157 } } }
I20260806 09:34:40.474901 22254 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5cfae47be6541ef8b7717525d447fd7 [sys.catalog]: This master's current role is: LEADER
I20260806 09:34:40.474115 22228 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5cfae47be6541ef8b7717525d447fd7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d5cfae47be6541ef8b7717525d447fd7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5cfae47be6541ef8b7717525d447fd7" member_type: VOTER last_known_addr { host: "127.20.113.62" port: 32933 } } peers { permanent_uuid: "18af5fcd3b6747348add713a9330a7b4" member_type: VOTER last_known_addr { host: "127.20.113.61" port: 45653 } } peers { permanent_uuid: "9eb948dc682344bb8c21303a508482c2" member_type: VOTER last_known_addr { host: "127.20.113.60" port: 37157 } } }
I20260806 09:34:40.475358 22228 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5cfae47be6541ef8b7717525d447fd7 [sys.catalog]: This master's current role is: LEADER
I20260806 09:34:40.501865 22260 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 09:34:40.504520 22260 catalog_manager.cc:1504] Loading token signing keys...
I20260806 09:34:40.511008 22311 catalog_manager.cc:1259] Loaded cluster ID: 362185097cca46cfae3831a65a238d31
I20260806 09:34:40.511128 22311 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 18af5fcd3b6747348add713a9330a7b4: loading cluster ID for follower catalog manager: success
I20260806 09:34:40.514810 22311 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 18af5fcd3b6747348add713a9330a7b4: acquiring CA information for follower catalog manager: success
W20260806 09:34:40.516085 22311 catalog_manager.cc:873] Not found: no TSK found in the system table: failed to prepare follower catalog manager, will retry
I20260806 09:34:40.529130 22260 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d5cfae47be6541ef8b7717525d447fd7: Generated new TSK 0
I20260806 09:34:40.530231 22260 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 09:34:40.711580 22221 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 09:34:40.696394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51178 (local address 127.20.113.62:32933)
0806 09:34:40.696639 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0806 09:34:40.696649 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 09:34:40.707050 (+ 10401us) server_negotiation.cc:407] Connection header received
0806 09:34:40.709831 (+ 2781us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:34:40.709838 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:34:40.709940 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:34:40.710025 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:34:40.711249 (+ 1224us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51178: BlockingRecv error: recv got EOF from 127.0.0.1:51178 (error 108)
Metrics: {"server-negotiator.queue_time_us":102}
W20260806 09:34:40.714329 22215 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 09:34:40.696131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36894 (local address 127.20.113.60:37157)
0806 09:34:40.696349 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0806 09:34:40.696356 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:34:40.708992 (+ 12636us) server_negotiation.cc:407] Connection header received
0806 09:34:40.712182 (+ 3190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:34:40.712188 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:34:40.712282 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:34:40.712387 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:34:40.714058 (+ 1671us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36894: BlockingRecv error: recv got EOF from 127.0.0.1:36894 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
W20260806 09:34:40.715992 22223 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 09:34:40.704611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41286 (local address 127.20.113.61:45653)
0806 09:34:40.704833 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0806 09:34:40.704839 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:34:40.709448 (+ 4609us) server_negotiation.cc:407] Connection header received
0806 09:34:40.712609 (+ 3161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:34:40.712616 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:34:40.712703 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:34:40.712772 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:34:40.715692 (+ 2920us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41286: BlockingRecv error: recv got EOF from 127.0.0.1:41286 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
W20260806 09:34:40.757637 22223 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 09:34:40.744510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41288 (local address 127.20.113.61:45653)
0806 09:34:40.744696 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0806 09:34:40.744702 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:34:40.755450 (+ 10748us) server_negotiation.cc:407] Connection header received
0806 09:34:40.755792 (+ 342us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:34:40.755798 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:34:40.755964 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:34:40.756053 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:34:40.757380 (+ 1327us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41288: BlockingRecv error: recv got EOF from 127.0.0.1:41288 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
W20260806 09:34:40.800822 22223 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 09:34:40.791775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41290 (local address 127.20.113.61:45653)
0806 09:34:40.791980 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0806 09:34:40.791986 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:34:40.798840 (+ 6854us) server_negotiation.cc:407] Connection header received
0806 09:34:40.799321 (+ 481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:34:40.799328 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:34:40.799427 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:34:40.799500 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:34:40.800536 (+ 1036us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41290: BlockingRecv error: recv got EOF from 127.0.0.1:41290 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
W20260806 09:34:40.849020 22223 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 09:34:40.839863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41302 (local address 127.20.113.61:45653)
0806 09:34:40.840049 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0806 09:34:40.840055 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:34:40.847031 (+ 6976us) server_negotiation.cc:407] Connection header received
0806 09:34:40.847513 (+ 482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:34:40.847520 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:34:40.847618 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:34:40.847691 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:34:40.848766 (+ 1075us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41302: BlockingRecv error: recv got EOF from 127.0.0.1:41302 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260806 09:34:40.904045 22223 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 09:34:40.894765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41310 (local address 127.20.113.61:45653)
0806 09:34:40.895033 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0806 09:34:40.895040 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:34:40.902040 (+ 7000us) server_negotiation.cc:407] Connection header received
0806 09:34:40.902570 (+ 530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:34:40.902583 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:34:40.902687 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:34:40.902762 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:34:40.903811 (+ 1049us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41310: BlockingRecv error: recv got EOF from 127.0.0.1:41310 (error 108)
Metrics: {"server-negotiator.queue_time_us":144}
W20260806 09:34:40.960431 22223 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 09:34:40.951034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41316 (local address 127.20.113.61:45653)
0806 09:34:40.951261 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0806 09:34:40.951269 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 09:34:40.958282 (+ 7013us) server_negotiation.cc:407] Connection header received
0806 09:34:40.958913 (+ 631us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:34:40.958919 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:34:40.959049 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:34:40.959123 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:34:40.960163 (+ 1040us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41316: BlockingRecv error: recv got EOF from 127.0.0.1:41316 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
W20260806 09:34:41.029726 22223 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 09:34:41.026532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41318 (local address 127.20.113.61:45653)
0806 09:34:41.026730 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0806 09:34:41.026736 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:34:41.027804 (+ 1068us) server_negotiation.cc:407] Connection header received
0806 09:34:41.028042 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:34:41.028048 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:34:41.028142 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:34:41.028259 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:34:41.029430 (+ 1171us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41318: BlockingRecv error: recv got EOF from 127.0.0.1:41318 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
W20260806 09:34:41.069201 22223 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 09:34:41.066513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41328 (local address 127.20.113.61:45653)
0806 09:34:41.066744 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0806 09:34:41.066751 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:34:41.067313 (+ 562us) server_negotiation.cc:407] Connection header received
0806 09:34:41.067604 (+ 291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:34:41.067611 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:34:41.067730 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:34:41.067888 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:34:41.068906 (+ 1018us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41328: BlockingRecv error: recv got EOF from 127.0.0.1:41328 (error 108)
Metrics: {"server-negotiator.queue_time_us":93}
W20260806 09:34:41.138808 22223 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 09:34:41.136673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41344 (local address 127.20.113.61:45653)
0806 09:34:41.136873 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0806 09:34:41.136880 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:34:41.137393 (+ 513us) server_negotiation.cc:407] Connection header received
0806 09:34:41.137573 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:34:41.137581 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:34:41.137695 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:34:41.137839 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:34:41.138573 (+ 734us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41344: BlockingRecv error: recv got EOF from 127.0.0.1:41344 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
W20260806 09:34:41.272290 22223 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 09:34:41.270510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41356 (local address 127.20.113.61:45653)
0806 09:34:41.270763 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0806 09:34:41.270769 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:34:41.271011 (+ 242us) server_negotiation.cc:407] Connection header received
0806 09:34:41.271186 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:34:41.271190 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:34:41.271262 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:34:41.271348 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 09:34:41.272071 (+ 723us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41356: BlockingRecv error: recv got EOF from 127.0.0.1:41356 (error 108)
Metrics: {"server-negotiator.queue_time_us":98}
I20260806 09:34:41.469959 22280 catalog_manager.cc:1259] Loaded cluster ID: 362185097cca46cfae3831a65a238d31
I20260806 09:34:41.470067 22280 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 9eb948dc682344bb8c21303a508482c2: loading cluster ID for follower catalog manager: success
I20260806 09:34:41.472636 22280 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 9eb948dc682344bb8c21303a508482c2: acquiring CA information for follower catalog manager: success
I20260806 09:34:41.474526 22280 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 9eb948dc682344bb8c21303a508482c2: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 09:34:41.518837 22311 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 18af5fcd3b6747348add713a9330a7b4: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 09:34:41.537705 22223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:41.532091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41372 (local address 127.20.113.61:45653)
0806 09:34:41.532282 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0806 09:34:41.532287 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 09:34:41.532657 (+ 370us) server_negotiation.cc:407] Connection header received
0806 09:34:41.532835 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:34:41.532839 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:34:41.532985 (+ 146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:34:41.533098 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 09:34:41.533520 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:41.534248 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:41.535560 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:41.536013 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:41.536108 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:34:41.536659 (+ 551us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 09:34:41.537197 (+ 538us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 09:34:41.537307 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 09:34:41.537401 (+ 94us) server_negotiation.cc:299] Negotiation successful
0806 09:34:41.537474 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 09:34:41.549973 22396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:41.543794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36898 (local address 127.20.113.60:37157)
0806 09:34:41.544314 (+ 520us) server_negotiation.cc:206] Beginning negotiation
0806 09:34:41.544323 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 09:34:41.544678 (+ 355us) server_negotiation.cc:407] Connection header received
0806 09:34:41.544930 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:34:41.544934 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:34:41.545107 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:34:41.545285 (+ 178us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 09:34:41.545711 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:41.546403 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:41.547761 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:41.548158 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:41.548290 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:34:41.548838 (+ 548us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 09:34:41.549347 (+ 509us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 09:34:41.549524 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0806 09:34:41.549594 (+ 70us) server_negotiation.cc:299] Negotiation successful
0806 09:34:41.549677 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":364,"thread_start_us":260,"threads_started":1}
I20260806 09:34:41.564940 22396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:41.556379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36904 (local address 127.20.113.60:37157)
0806 09:34:41.556596 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0806 09:34:41.556604 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 09:34:41.558883 (+ 2279us) server_negotiation.cc:407] Connection header received
0806 09:34:41.559123 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:34:41.559128 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:34:41.559199 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:34:41.559296 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 09:34:41.559754 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:41.560757 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:41.562766 (+ 2009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:41.563365 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:41.563507 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:34:41.564152 (+ 645us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 09:34:41.564248 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 09:34:41.564371 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0806 09:34:41.564548 (+ 177us) server_negotiation.cc:299] Negotiation successful
0806 09:34:41.564624 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 09:34:41.567277 22223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:41.555822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41386 (local address 127.20.113.61:45653)
0806 09:34:41.556021 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0806 09:34:41.556028 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 09:34:41.561134 (+ 5106us) server_negotiation.cc:407] Connection header received
0806 09:34:41.561393 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:34:41.561400 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:34:41.561490 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:34:41.561558 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 09:34:41.562316 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:41.563236 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:41.565354 (+ 2118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:41.565883 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:41.566024 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:34:41.566540 (+ 516us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 09:34:41.566672 (+ 132us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 09:34:41.566771 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 09:34:41.566923 (+ 152us) server_negotiation.cc:299] Negotiation successful
0806 09:34:41.567016 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 09:34:41.567683 22409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 09:34:41.555796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51184 (local address 127.20.113.62:32933)
0806 09:34:41.560678 (+ 4882us) server_negotiation.cc:206] Beginning negotiation
0806 09:34:41.560684 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 09:34:41.560702 (+ 18us) server_negotiation.cc:407] Connection header received
0806 09:34:41.560829 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 09:34:41.560838 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 09:34:41.561073 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 09:34:41.561255 (+ 182us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 09:34:41.561913 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:41.562929 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:41.565049 (+ 2120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 09:34:41.565695 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 09:34:41.565834 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 09:34:41.566403 (+ 569us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 09:34:41.566945 (+ 542us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 09:34:41.567050 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 09:34:41.567283 (+ 233us) server_negotiation.cc:299] Negotiation successful
0806 09:34:41.567483 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4766,"thread_start_us":4667,"threads_started":1}
I20260806 09:34:41.572199 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 21894
I20260806 09:34:41.573524 21987 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:34:41.688230 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 21894
W20260806 09:34:41.719805 22034 connection.cc:535] server connection from 127.0.0.1:41238 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 09:34:41.720604 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 22003
I20260806 09:34:41.721725 22095 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:34:41.828835 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 22003
I20260806 09:34:41.860069 20932 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu pid 22114
I20260806 09:34:41.861290 22205 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 09:34:41.968662 20932 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKR4MEC/build/asan/bin/kudu with pid 22114
2026-08-06T09:34:41Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (5046 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (10205 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (386165 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST